Question: examples/simultaneous_translation doesn't work

[mako2sake]

❓ Questions and Help

What is your question?

To reproduce, I'm tring to do below instruction. But it doesn't work.

https://github.com/facebookresearch/fairseq/tree/iwslt2023/examples/simultaneous_translation

Ref: https://iwslt.org/2023/simultaneous

Error mesage

Traceback (most recent call last):
  File "${WDIR}/.venv/bin/fairseq-train", line 8, in <module>
    sys.exit(cli_main())
  File "${WDIR}/fairseq/fairseq_cli/train.py", line 574, in cli_main
    distributed_utils.call_main(cfg, main)
  File "${WDIR}/fairseq/fairseq/distributed/utils.py", line 404, in call_main
    main(cfg, **kwargs)
  File "${WDIR}/fairseq/fairseq_cli/train.py", line 96, in main
    model = task.build_model(cfg.model)
  File "${WDIR}/fairseq/fairseq/tasks/speech_to_text.py", line 170, in build_model
    return super(SpeechToTextTask, self).build_model(args, from_checkpoint)
  File "${WDIR}/fairseq/fairseq/tasks/fairseq_task.py", line 691, in build_model
    model = models.build_model(args, self, from_checkpoint)
  File "${WDIR}/fairseq/fairseq/models/__init__.py", line 106, in build_model
    return model.build_model(cfg, task)
  File "${WDIR}/fairseq/fairseq/models/speech_to_text/xm_transformer.py", line 646, in build_model
    decoder_args_dict = cls.get_decoder_args_from_checkpoint(ckpt["cfg"])
KeyError: 'cfg'

This time, ckpt.keys() is dict_keys(['args', 'model', 'optimizer_history', 'extra_state', 'last_optimizer_state']).

ckpt is loaded using torch.load from --load_pretrained_decoder_from option path.

so maybe I think the loaded modei is bad?

--load_pretrained_decoder_from value is model.pt as below instruction.

For pretrained decoder, we used an mBART model opensourced by the original mBART paper. Download and extract the model to ${MUSTC_ROOT}/en-${TARGET_LANG}/model.pt, the dict to ${MUSTC_ROOT}/en-${TARGET_LANG}/dict.txt and the sentencepiece model to ${MUSTC_ROOT}/en-${TARGET_LANG}/sentence.bpe.model

What's your environment?

• fairseq Version (e.g., 1.0 or main): 0.12.2
• PyTorch Version (e.g., 1.0): 2.0
• OS (e.g., Linux):Ubuntu 20.04
• How you installed fairseq (pip, source): source
• Build command you used (if compiling from source): pip install --editable ./
• Python version: 3.8
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

[bhaddow]

Hi

I see the same error. Making the change

+++ b/fairseq/models/speech_to_text/xm_transformer.py
@@ -643,7 +643,7 @@ class XMTransformerModel(FairseqEncoderDecoderModel):
         base_architecture(args)
         if getattr(args, "load_pretrained_decoder_from", None) is not None:
             ckpt = torch.load(getattr(args, "load_pretrained_decoder_from", None))
-            decoder_args_dict = cls.get_decoder_args_from_checkpoint(ckpt["cfg"])
+            decoder_args_dict = cls.get_decoder_args_from_checkpoint(ckpt)
             args = cls.override_decoder_args(args, decoder_args_dict)

fixes this error. But then there are other configuration errors.

best Barry

[yusuke1997]

@bhaddow Hi, Thank you for your advice. I face some (configuration) errors. How do you solve it?

[yusuke1997]

@xutaima Hi, we have some troubles with the baseline of iwslt2023. I follow the instructions in the README, but it does not work. Also, the iwslt2024 baseline is the same, but it is not maintained.

For example, when fixing https://github.com/facebookresearch/fairseq/issues/5083#issuecomment-1514713597, I caught following error:

omegaconf.errors.MissingMandatoryValue: Missing mandatory value: model.w2v_path
	full_key: model.w2v_path
	reference_type=Any
	object_type=dict

I tried to fix it step-by-step, but I caught the other error... We need a fundamental solution.

best,