anand-nv

Hi @dineshtripathi30 the issue is with Inverse Text Normalization. You could generate new tokenizer and verbalizer files from https://github.com/NVIDIA/NeMo-text-processing/tree/en_tech and use them in your Riva server build. This should resolve...

We haven't been able to reproduce this. Have you tried this on another machine?

Can you provide the steps your are following to evaluate. Providing Python scripts and sparrowhawk code snippets used for benchmarking and performing ITN/TN would be useful.

Do you have the "actual time estimates" for the C++ implementation `normalizer_main.cc` ?

Are you using the Dockerfile provided here for building sparrowhawk. If so can you try adding `'CXXFLAGS'` and `'CFLAGS' ` to `./configure` and rebuild the docker. ` ./configure CFLAGS='-g -O2...

The docker container used for sparrowhawk is not optimized for production. You can build your own docker container and compile openfst-1.7.9, thrax-1.3.4 and sparrowhawk (https://github.com/anand-nv/sparrowhawk/tree/nemo_tests) with `CXXFLAGS=-g -O2` The time...

The encoder is a [fastconformer encoder](https://arxiv.org/abs/2305.05084). Initial attempts at implementing it using TRT-LLM layers resulted in poorer perf than onnx->trt.

> I'm working on my own attempts of optimizing TRT versions of Conformer models, if you can share your initial attempts I can help and contribute back my results, I...

@juney-nvidia - rebase done

@juney-nvidia @Shixiaowei02 Can this be reviewed and merged?