Issue about performance

[Shengqiang-Li]

I have finished training and decoding in AISHELL-1 dataset and got cer=12.4% in test set，and i found that my model.json which uses the default config is different from the one of Streaming_transformer-chunk32 with ESPnet Conv2d Encoder. It seems that my model lacks something,such as adaptive decoder.Can you release the result in AISHELL-1？

[MarkWuNLP]

I have finished training and decoding in AISHELL-1 dataset and got cer=12.4% in test set，and i found that my model.json which uses the default config is different from the one of Streaming_transformer-chunk32 with ESPnet Conv2d Encoder. It seems that my model lacks something,such as adaptive decoder.Can you release the result in AISHELL-1？

Hi, This is Yu Wu, the mentor of Chengyi at MSRA. Chengyi is on vacation now, so she may not respond to you in a short time. As far as I known, Chengyi didn't try streaming SR on AISHELL-1 but her planning is to do it later. From my perspective, trigger attention algorithm is very tricky, so we have to do parameter tuning on different test sets to balance attention weight, ctc weight, and LM weight. I tried the algorithm on Microsoft internal dataset, and find the hyper-parameter is not a good setting for that dataset. I obtained absolute 10 WER gain by changing hyper-parameter

[Shengqiang-Li]

Thank you very much.

[MarkWuNLP]

Thank you very much.

Chengyi told me her model achieves 6.0 on AISELL-1 yesterday with a conformer architecture as an encoder.

[cywang97]

I have finished training and decoding in AISHELL-1 dataset and got cer=12.4% in test set，and i found that my model.json which uses the default config is different from the one of Streaming_transformer-chunk32 with ESPnet Conv2d Encoder. It seems that my model lacks something,such as adaptive decoder.Can you release the result in AISHELL-1？

Hi, I've done several experiments on AISHELL-1 dataset and got the following results on test set: Offline Transformer 5.6 Offline Transformer+lm 5.4 Streaming Transformer-chunk16 6.6 Offline Conformer 4.9 Streaming Conformer-chunk16 6.2

I will release my code for Conformer and the pretrained model soon.

[huangzj421]

I have finished training and decoding in AISHELL-1 dataset and got cer=12.4% in test set，and i found that my model.json which uses the default config is different from the one of Streaming_transformer-chunk32 with ESPnet Conv2d Encoder. It seems that my model lacks something,such as adaptive decoder.Can you release the result in AISHELL-1？

Hi, I've done several experiments on AISHELL-1 dataset and got the following results on test set: Offline Transformer 5.6 Offline Transformer+lm 5.4 Streaming Transformer-chunk16 6.6 Offline Conformer 4.9 Streaming Conformer-chunk16 6.2

I will release my code for Conformer and the pretrained model soon.

May i ask how is it going for Streaming Conformer-chunk? I have been waiting for a few months, thanks!

[Some-random]

I have finished training and decoding in AISHELL-1 dataset and got cer=12.4% in test set，and i found that my model.json which uses the default config is different from the one of Streaming_transformer-chunk32 with ESPnet Conv2d Encoder. It seems that my model lacks something,such as adaptive decoder.Can you release the result in AISHELL-1？

Hi, I've done several experiments on AISHELL-1 dataset and got the following results on test set: Offline Transformer 5.6 Offline Transformer+lm 5.4 Streaming Transformer-chunk16 6.6 Offline Conformer 4.9 Streaming Conformer-chunk16 6.2

I will release my code for Conformer and the pretrained model soon.

I'm also looking forward to use these models, have them been released?

[cywang97]

Sorry for the late reply. For streaming Conformer, you can simply set the conv module in the encoder to casual conv and the self-attention layers are as same as in streaming Transformer. I have updated the code for conformer and I will release my model next week.

[TIFOSI528]

I have finished training and decoding in AISHELL-1 dataset and got cer=12.4% in test set，and i found that my model.json which uses the default config is different from the one of Streaming_transformer-chunk32 with ESPnet Conv2d Encoder. It seems that my model lacks something,such as adaptive decoder.Can you release the result in AISHELL-1？

Hi, I've done several experiments on AISHELL-1 dataset and got the following results on test set: Offline Transformer 5.6 Offline Transformer+lm 5.4 Streaming Transformer-chunk16 6.6 Offline Conformer 4.9 Streaming Conformer-chunk16 6.2

I will release my code for Conformer and the pretrained model soon.

Where can I find the configs for these Aishell-1 experiments? Many thanks.