n-best rescore with transformer lm

[glynpu]

Wer results of this pr (by loaded models from espnet model zoo):

test-clean 2.43% 
test-other 5.79%

This pr implements following procedure with models from espnet model zoo:

Added benefits by loading espnet trained conformer encoder model with equivalent snowfall model definition:

1. identify differences of conformer implementation between espnet and snowfall. As shown in snowfall/models/conformer.py, snowfall only scaling q; while espnet scale attn_outout_weights.
2. espnet conformer has an extra layer_norm after encoder

Also, the loaded espnet transformer lm could be used as a baseline for snowfall lm training tasks.

[danpovey]

Great!! I assume the modeling units are BPE pieces? I think a good step towards resolving the difference would be to train (i) a CTC model (ii) a LF-MMI model using those same BPE pieces.

[glynpu]

Great!! I assume the modeling units are BPE pieces? I think a good step towards resolving the difference would be to train (i) a CTC model (ii) a LF-MMI model using those same BPE pieces.

Yes, the modeling units are 5000 tokens including "<blank>". I will do the suggested experiments.

[danpovey]

Thanks! You may run into memory problems. Fangjun recently committed a code change that can be used to work around something related to that, though. We need to make sure our recipes can run for those kinds of sizes anyway.

On Tue, May 25, 2021 at 10:21 AM LIyong.Guo @.***> wrote:

Great!! I assume the modeling units are BPE pieces? I think a good step towards resolving the difference would be to train (i) a CTC model (ii) a LF-MMI model using those same BPE pieces.

Yes, the modeling units are 5000 tokens including . I will do the suggested experiments.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/k2-fsa/snowfall/pull/201#issuecomment-847480709, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO2ABWX6JSLSM35IIELTPMCSRANCNFSM45NKCFJQ .