lance icon indicating copy to clipboard operation
lance copied to clipboard

Published 20 hours ago • verified publisher icon lancedb

Limit parallelism in `Dataset.cleanup_old_versions`

Open tonyf opened this issue 5 months ago • 7 comments

Running into s3 rate limits when trying to cleanup a very large dataset with dataset.cleanup_old_versions. Can't seem to control this via LANCE_IO_THREADS

tonyf avatar Aug 28 '24 21:08 tonyf