LLM-Finetuning-Toolkit
LLM-Finetuning-Toolkit copied to clipboard
Allow custom train/test datasets
Is your feature request related to a problem? Please describe.
I'm working on a problem that requires me to split my data in a specific way (base on dates). Right now the config only allows for a single dataset to be provided and it internally does a train-test split based on the values provided for the test_size and train_size parameters.
Describe the solution you'd like Ideally, an option to specify paths to both train and test data.
Describe alternatives you've considered The alternative would be to add in support for other types of data splitting which I don't think makes sense for this repo to include.
Additional context None