Pre-trained Machine Translation Models of Korean from/to ECJ

Pre-trained models are beautiful. They save your time, energy and/or money. You can obtain several pre-trained machine translation models for mostly European languages here. In this project, I add six other models: Korean <-> English, Chinese, Japanese as I failed to find publicly available ones. Not surprisingly, the biggest challenge in training NMT models for those language pairs is the lack of large parallel corpora. I decided to use both public data (OpenSubtitles) and private data) to overcome the difficulties. Overall, each of their performance may not so impressive, but you can keep training it with your own data, if necessary.

Requirements

• python >=3.6
• pytorch >=1.0
• Fairseq

Data

Model

• Transformer Base

Vocabulary and tokenization

• Click the links to download the pretrained models and vocabulary files.

Pre-trained models and their performance

• Evaluation is based on the tokenization tools such as Mecab-ko (ko), NLTK punct (en), pkuseg (zh), and MeCab (ja).)

Finetuning Examples

echo "ko -> en"
python -m torch.distributed.launch  --nproc_per_node 8 FAIRSEQ/train.py    ko-en-bin --arch transformer       --optimizer adam --lr 0.0005 --label-smoothing 0.1 --dropout 0.3       --max-tokens 4000 --min-lr '1e-09' --lr-scheduler inverse_sqrt       --weight-decay 0.0001 --criterion label_smoothed_cross_entropy       --max-epoch 80 --warmup-updates 4000 --warmup-init-lr '1e-07'    --adam-betas '(0.9, 0.98)'   --save-dir train/ko-en/ckpt  --save-interval 1 --restore-file checkpoint77.pt