unitxt
unitxt copied to clipboard
LoadFromIBMCos does not work if datafile has '/' in name
data_dir=“tuning-data-cleared/ceramic/mixtures_02.26.2024/code/dolphin_coder”, data_files={“train”: “train/part0.jsonl”, “test”: “val/part0.jsonl”},
This causes failures.
Possible solution:
- Raise an error if '/' in data_files, but allow fusing of datasource (see #707)
- Handle '/' in paths. (simpler for users)