distilabel
distilabel copied to clipboard
[BUG] `make_generator_step` can fail when setting the `_dataset_info` internally
Describe the bug
When a loader step is created using make_generator_step
and something fails, we cannot control it right now.
A case that's happened is the code trying to load a dataset and failing (even though the dataset has already been downloaded).
Also we need to call the load
method from the internal step: super(loader.__class__, loader).load()
To Reproduce Code to reproduce
Expected behaviour A clear and concise description of what you expected to happen.
Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- Package version:
- Python version:
Additional context Add any other context about the problem here.