sherpa-onnx icon indicating copy to clipboard operation
sherpa-onnx copied to clipboard

fast-beam-search support for sherpa-onnx

Open uni-manjunath-ke opened this issue 2 years ago • 10 comments

I found that the fast-beam-search decoding is currently not supported in sherpa-onnx. Is this activity is planned for future? If yes, when can this be expected (timeline)?

In specific, do you plan to support fast-beam-search-with-lg. Because, currently, the fast-beam-search-with-lg is not supported by zipformer/streaming-decode.py. Thanks

uni-manjunath-ke avatar Sep 26 '23 11:09 uni-manjunath-ke

fast_beam_search depends on k2. If you want to use fast_beam_search, please use k2-fsa/sherap.

We are going to support HLG decoding in sherpa-onnx. If you want to use a lexicon or/and an n-gram LM in decoding, then you may find HLG decoding interesting.

Please also have a look at https://github.com/k2-fsa/icefall/pull/1275

csukuangfj avatar Sep 26 '23 11:09 csukuangfj

Sure, Thanks. Ya, we are interested in HLG. Hoping to see this HLG code at https://github.com/k2-fsa/icefall/pull/1275 merged soon and available for usage.

uni-manjunath-ke avatar Sep 26 '23 11:09 uni-manjunath-ke

Hi @csukuangfj , Just wanted to confirm, if the HLG code at https://github.com/k2-fsa/icefall/pull/1275 is only for CTC, or it can be used with zipformers as well. Could you please confirm. Because, we are interested in Zipformer models for now.

uni-manjunath-ke avatar Sep 28 '23 09:09 uni-manjunath-ke

Currently, I see in https://github.com/k2-fsa/icefall/pull/1275 that HLG support is added for ICefall, When can we expect this support to be ported for sherpa-onnx? or Is that already ported to Sherpa-onnx? Thanks

uni-manjunath-ke avatar Sep 28 '23 09:09 uni-manjunath-ke

Currently, I see in k2-fsa/icefall#1275 that HLG support is added for ICefall, When can we expect this support to be ported for sherpa-onnx? or Is that already ported to Sherpa-onnx? Thanks

please see https://github.com/k2-fsa/sherpa-onnx/pull/349

We will finish it in two weeks.


if the HLG code at https://github.com/k2-fsa/icefall/pull/1275 is only for CTC

Yes, you are right. It is only for CTC.


it can be used with zipformers as well

Zipformer is a kind of neural network, while ctc is a kind of loss function. They are two different things. If you train a zipformer using CTC loss, then you can use HLG decoding with zipformer. If you train a zipformer using transducer loss, then you cannot use HLG decoding with zipformer.

csukuangfj avatar Sep 29 '23 14:09 csukuangfj

Currently, I see in k2-fsa/icefall#1275 that HLG support is added for ICefall, When can we expect this support to be ported for sherpa-onnx? or Is that already ported to Sherpa-onnx? Thanks

@uni-manjunath-ke

https://github.com/k2-fsa/sherpa-onnx/pull/349

The C++ part is usable now.

csukuangfj avatar Oct 07 '23 14:10 csukuangfj

Thank you.

uni-manjunath-ke avatar Oct 10 '23 08:10 uni-manjunath-ke