llm-foundry icon indicating copy to clipboard operation
llm-foundry copied to clipboard

Why does convert_dataset_json.py only support 'train' for the --split argument?"

Open sysusicily opened this issue 2 years ago • 1 comments

image

sysusicily avatar May 26 '23 08:05 sysusicily

From the trace you posted, it looks like this is an issue with your dataset and/or HuggingFace's datasets library—the error is being raised by datasets/arrow_reader.py. My hunch is that there's no val split for your dataset, though I could be mistaken.

growlix avatar Jun 13 '23 23:06 growlix