Restoring from pretrained checkpoint is broke for "stt_ua_fastconformer_hybrid_large_pc"

[taras-sereda]

Hi folks! I'm trying to restore EncDecHybridRNNTCTCModel from checkpoint trained on Ukrainian

In [4]: from nemo.collections.asr.models import EncDecHybridRNNTCTCBPEModel
In [5]: model = EncDecHybridRNNTCTCBPEModel.from_pretrained("stt_ua_fastconformer_hybrid_large_pc")

model can't be restored because of keyword argument conv_pointwise_type

InstantiationException: Error in call to target 'nemo.collections.asr.modules.conformer_encoder.ConformerEncoder':
TypeError("ConformerEncoder.__init__() got an unexpected keyword argument 'conv_pointwise_type'")

Search through NeMo repository I only found conv_pointwise_type keyword in one PR, and nowhere in the code base, that looks suspicious. Is there any recommendation how one would make that checkpoint compatible?

Env:

• Nemo version: nemo_toolkit==1.23.0
• OS: Ubuntu 22.04

[taras-sereda]

I figured how to load the parameters, though with the hack.

1. I obtained model's config by doing this:

ckpt_name = "stt_ua_fastconformer_hybrid_large_pc"
cfg = nemo_asr.models.EncDecHybridRNNTCTCBPEModel.from_pretrained(model_name=ckpt_name, return_config=True)
omegaconf.OmegaConf.save(cfg, f="cfg.yaml")

2. then I commented out the conv_pointwise_type attribute of encoder's parameters.

3. loaded model by providing a patched config:

model = nemo_asr.models.EncDecHybridRNNTCTCBPEModel.from_pretrained(
    model_name=ckpt_path,
    override_config_path="./cfg_modified.yaml",

The above hack works, though I wonder if the loaded model works correctly? Also it will be good to update the official checkpoints in the NGC registry.

Same issue happens to stt_ru_fastconformer_hybrid_large_pc russian checkpoint.

[titu1994]

Thank you for reporting the issue and the fix ! We'll take a look soon. I think we updated the huggingface version but not NGC

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[kzelias]

Still relevant for https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_ru_fastconformer_hybrid_large_pc docker: nvcr.io/nvidia/nemo:24.01.speech

And this model has a different tokenizer from the one in ngc. https://huggingface.co/nvidia/stt_ru_fastconformer_hybrid_large_pc/tree/main