torchtune icon indicating copy to clipboard operation
torchtune copied to clipboard

text_completion_dataset removed?

Open wiiiktor opened this issue 7 months ago • 6 comments

This line in my custom recipe does not work (the only one that I have added): from torchtune.datasets import text_completion_dataset

When I run tune, the message is: ImportError: cannot import name 'text_completion_dataset' from 'torchtune.datasets' (/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/torchtune/datasets/init.py)

Also, I have noticed that a similar import line: from torchtune.datasets import ConcatDataset is not present in a file that I receive after using command: tune cp lora_finetune_distributed ./custom_lora_finetune_distributed.py

To me, it looks like the torchtune.datasets library has been removed or updated. how can I use text_completion_dataset configuration now?

wiiiktor avatar Jul 03 '24 00:07 wiiiktor