Can I finetune custom trained model?

[varna9000]

I have a dataset of ~2GB. I can't fit it in my GPU, so I decided to split the dataset into chunks of 500MB If I train gpt-2 from scratch on first chunk of the dataset, can I use the already trained custom model for fine-tuning on the rest of the dataset chunks?

I appreciate the above problem might not be specific to aitextgen, but I can't find another workaround for fitting the big dataset into GPU.

[tientr]

You can, however GPT intend to bias toward to last chunk of your dataset. To avoid that, you can use merge_datasets().