OpenNMT-kpg-release icon indicating copy to clipboard operation
OpenNMT-kpg-release copied to clipboard

Published 20 hours ago • verified publisher icon memray

Fine tune on custom dataset

Open rafikg opened this issue 3 years ago • 2 comments

Hi @memray Is there any readme or notebook to fine-tune the model on custom dataset?

rafikg avatar Feb 22 '22 19:02 rafikg

Please stay tuned as we are preparing for a tutorial at ECIR. A new codebase will be released and it can support fine-tuning better. It supports more pretrained models and datasets. New checkpoints will be released as well. Appreciate your patience!

Rui

memray avatar Feb 26 '22 13:02 memray

Dear Rui, any updates on this.

  • Many notebooks rely on /data/keyphrase which does not exit in the latest master branch
  • Would love to see such a demo to fine tune models on our own data?

Regards!

ahadda5 avatar Jun 13 '22 12:06 ahadda5

Hi @rafikg and @ahadda5 , I uploaded datasets I used for training at huggingface. Following the same json format and config files here, you would be able to train models with your own data. Notebook is not a good way for training since it doesn't work well with multiple GPUs.

memray avatar Oct 29 '22 09:10 memray