torchtune
torchtune copied to clipboard
text_completion_dataset removed?
This line in my custom recipe does not work (the only one that I have added): from torchtune.datasets import text_completion_dataset
When I run tune, the message is: ImportError: cannot import name 'text_completion_dataset' from 'torchtune.datasets' (/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/torchtune/datasets/init.py)
Also, I have noticed that a similar import line: from torchtune.datasets import ConcatDataset is not present in a file that I receive after using command: tune cp lora_finetune_distributed ./custom_lora_finetune_distributed.py
To me, it looks like the torchtune.datasets library has been removed or updated. how can I use text_completion_dataset configuration now?