Minerva icon indicating copy to clipboard operation
Minerva copied to clipboard

Published 20 hours ago • verified publisher icon Pale-Blue-Dot-97

Bug in caching dataset with existing file error in distributed computing

Open Pale-Blue-Dot-97 opened this issue 7 months ago • 1 comments

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Use a distributed computing setup
  2. Ensure cache==True for make_dataset
  3. See error

Expected behavior No error. If the dataset is not already cached, it should be created then cached under the unique hash. If it exists, the hash should be recognised and the dataset loaded.

Environment (please complete the following information):

  • OS: Ubuntu
  • Version: 0.27.0
  • Python Version: 3.11
  • Packages: pytorch==2.1.2

Pale-Blue-Dot-97 avatar Jan 23 '24 10:01 Pale-Blue-Dot-97