nemo2riva: `AttributeError: 'EncDecRNNTBPEModel' object has no attribute 'input_example'`

[lifefeel]

trafficstars

Describe the bug

I'm trying to convert .nemo to .riva, but I'm getting message AttributeError: 'EncDecRNNTBPEModel' object has no attribute 'input_example. Converting the Conformer-CTC model is OK but the Conformer-Transducer model cannot.

run command:

nemo2riva --out model_conformer_transducer.riva Conformer-Transducer-BPE.nemo

got following error:

[NeMo W 2022-08-08 17:31:20 schema:118] Validation schema not found for nemo.collections.asr.models.EncDecRNNTBPEModel.
    That means Riva does not yet support a pipeline for this network and likely will not work with it.
[NeMo I 2022-08-08 17:31:20 artifacts:65] Found model at ./model_weights.ckpt
[NeMo I 2022-08-08 17:31:20 artifacts:143] Retrieved artifacts: dict_keys(['08c8ba770083451b8aad07ed7310d7ef_tokenizer.model', '596cc7598fe44de9a3f4db5f5f5b0a3d_tokenizer.vocab', 'e9aec1bd43144fd3a60e65bee8ae73f2_vocab.txt', 'model_config.yaml'])
[NeMo I 2022-08-08 17:31:20 convert:78] Successfully exported model to model_graph.onnx, saving to model_conformer_transducer.riva ...
[NeMo I 2022-08-08 17:31:20 cookbook:93] Exporting model with autocast=False
[NeMo E 2022-08-08 17:31:20 cookbook:122] ERROR: Export failed. Please make sure your NeMo model class (nemo.collections.asr.models.EncDecRNNTBPEModel) has working export() and that you have the latest NeMo package installed with [all] dependencies.
Traceback (most recent call last):
  File "/opt/conda/bin/nemo2riva", line 8, in <module>
    sys.exit(nemo2riva())
  File "/opt/conda/lib/python3.8/site-packages/nemo2riva/cli/nemo2riva.py", line 56, in nemo2riva
    Nemo2Riva(args)
  File "/opt/conda/lib/python3.8/site-packages/nemo2riva/convert.py", line 79, in Nemo2Riva
    save_archive(model=model, save_path=riva_out, cfg=cfg, artifacts=artifacts, metadata=manifest['metadata'])
  File "/opt/conda/lib/python3.8/site-packages/nemo2riva/cookbook.py", line 123, in save_archive
    raise e
  File "/opt/conda/lib/python3.8/site-packages/nemo2riva/cookbook.py", line 96, in save_archive
    input_example = model.input_module.input_example(**in_args)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1204, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'EncDecRNNTBPEModel' object has no attribute 'input_example'

Steps/Code to reproduce bug

1. I trained using the config file from:
   • https://github.com/NVIDIA/NeMo/blob/main/examples/asr/conf/conformer/conformer_transducer_bpe.yaml
2. When training, .nemo file is saved every epoch.
3. I use .nemo and run it like below:
   • nemo2riva --out model_conformer_transducer.riva Conformer-Transducer-BPE.nemo

Expected behavior

Convert .nemo to .riva file using nemo2riva

Environment overview (please complete the following information)

• Environment location: Docker
• Method of NeMo install: already installed in Docker image
• If method of install is [Docker], provide docker pull & docker run commands used
  • docker pull nvcr.io/nvidia/nemo:22.05

docker run command:

docker run --runtime=nvidia -it -d --rm \
--gpus '"device=0"' \
-v /home/workspace/NeMo:/root/NeMo \
--shm-size=16g --ulimit memlock=-1 --ulimit stack=67108864 \
--name=nemo_22.05 nvcr.io/nvidia/nemo:22.05

Environment details

• Docker image : nvcr.io/nvidia/nemo:22.05

I installed additional Python package:

• nemo2riva 2.3.0

If NVIDIA docker image is used you don't need to specify these. Otherwise, please provide:

• OS version
• PyTorch version
• Python version

Additional context

Add any other context about the problem here. Example: GPU model

[titu1994]

@borisfom could you take a look ?

[VahidooX]

The support of Transducer models is not added to Riva yet. However you may convert these models to ONNX if you want.

[catalwaysright]

The support of Transducer models is not added to Riva yet. However you may convert these models to ONNX if you want.

How can convert it to ONNX since the export.py also need input_example?

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[XuesongYang]

@catalwaysright Here shows how to convert to ONNX: https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/core/export.html?highlight=onnx

[titu1994]

RNNT in particular is a bit more involved. You can use this script here to export and eval for comparison between nemo and onnx models - https://github.com/NVIDIA/NeMo/blob/main/examples/asr/export/transducer/infer_transducer_onnx.py