Delta LM model checkpoint

[ndenStanford]

I have explored the possibility of using the DeltaLM model in performing multilingual abstractive summarization. The paper has demonstrated a SOTA performance in this task. Would you mind letting me know if the trained checkpoint for such a task is available? I appreciate your attention to this matter.

[shumingma]

Hi, thanks for the interest in our work. Unfortunately, we don't have any plan to release the checkpoints finetuned on the downstream tasks. Yet, both the pretrained DeltaLM and the dataset in our paper are public available, so it should be not difficult to reproduce the results in the paper.

[yugaljain1999]

Hey @shumingma I have one small query though, Can you pls tell me where I can find name of all languages(100+ as per paper) on which deltalLM is pretrained? As I want to finetune deltaLM for translation of below languages to English..

German Russian Spanish Chinese French Finnish Swedish Polish Dutch Portuguese Slovak Hungarian Indonesian Italian Danish Romanian Serbian Norwegian Czech Slovenian

Looking forward to your response.. Thanks

[shumingma]

Hi @yugaljain1999, DeltaLM supports the same languages as InfoXLM. You can find those languages in the appendix of InfoXLM paper.

[yugaljain1999]

@shumingma

EDITED I just found a way to generate translation in real time where input will be plain text and return translation, but I just observed strange thing here, translation we were getting during evaluation on test set(where pre-processed .bin format test data as an input) are good but when we use interactive.py script to generate translation from plain text, it doesn't give good translation, even there was some repeated and noisy words also which we didn't observe in evaluation(using fairseq generate.py). So how can we generate correct translation using interactive.py(same we are getting using generate.py evaluation)?

Here is the command I am using python interactive.py /data/yugaljain/translation-pipeline/deltalm_setup/unilm/deltalm/tmp_data/wmt18_de_en/wmt18.bin --input - --path tmp_data/wmt18_de_en/checkpoints_deltalm_base/checkpoint_best.pt --source-lang de --target-lang en --beam 5 --remove-bpe sentencepiece

Looking forward to your response.

Thanks