distilabel icon indicating copy to clipboard operation
distilabel copied to clipboard

[BUG] `make_generator_step` can fail when setting the `_dataset_info` internally

Open plaguss opened this issue 5 months ago • 0 comments

Describe the bug When a loader step is created using make_generator_step and something fails, we cannot control it right now. A case that's happened is the code trying to load a dataset and failing (even though the dataset has already been downloaded). Also we need to call the load method from the internal step: super(loader.__class__, loader).load()

To Reproduce Code to reproduce


Expected behaviour A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • Package version:
  • Python version:

Additional context Add any other context about the problem here.

plaguss avatar Sep 02 '24 13:09 plaguss