Use CTC loss as an auxiliary loss for transducer training.

[csukuangfj]

[csukuangfj]

Here are the results of this PR.

I tried two models:

• exp-L: It is from https://github.com/k2-fsa/icefall/blob/master/egs/librispeech/ASR/RESULTS.md#large, which has about 118 M parameters
• exp: It is from https://github.com/k2-fsa/icefall/blob/master/egs/librispeech/ASR/RESULTS.md#baseline-2, which has about 88 M parameters

It shows:

• Add CTC loss as an auxiliary loss in RNN-T training is helpful
• ctc loss scale 0.20 is a good value for both settings

---

Results for exp-L

The WERs in the cell are for test_clean/test_other

ctc_loss_scale	greedy_search	modified_beam_search	fast_beam_search	comment
0.0 (baseline)	2.39/5.57	2.35/5.50	2.38/5.50	epoch-39-avg-7
0.10	2.40/5.29	2.39/5.26	2.39/5.27	epoch-30-avg-14
0.20	2.29/5.36	2.27/5.35	2.28/5.31	epoch-38-avg-23
0.25	2.31/5.52	2.28/5.52	2.30/5.53	epoch-30-avg-15
0.30	2.33/5.40	2.30/5.35	2.32/5.34	epoch-37-avg-21
0.40	2.32/5.38	2.28/5.37	2.33/5.37	epoch-40-avg-28
0.50	2.33/5.55	2.32/5.55	2.35/5.55	epoch-29-avg-14

Results for exp

ctc_loss_scale	greedy_search	modified_beam_search	fast_beam_search	comment
0.0 (baseline)	2.41/5.70	2.41/5.69	2.41/5.69	epoch-31-avg-17
0.10	2.41/5.59	2.42/5.55	2.42/5.57	epoch-39-avg-25
0.20	2.34/5.60	2.33/5.59	2.34/5.56	epoch-38-avg-16
0.25	2.43/5.72	2.39/5.67	2.39/5.66	epoch-37-avg-20
0.30	2.42/5.63	2.40/5.57	2.40/5.57	epoch-31-avg-13
0.40	2.37/5.67	2.33/5.66	2.34/5.66	epoch-39-avg-23
0.50	2.39/5.85	2.40/5.82	2.39/5.82	epoch-29-avg-13

[yaozengwei]

I use the above trained models for ctc decoding and also get promising results.

ctc-loss-scale	ctc-decoding	1best	Nbest	Nbest-rescoring	whole-lattice-rescoring	comment
0.10	2.66 & 5.92	2.71 & 5.4	2.71 & 5.39	2.69 & 5.3	2.68 & 5.29	epoch-40-avg-14
0.20	2.54 & 5.87	2.67 & 5.6	2.67 & 5.6	2.61 & 5.45	2.6 & 5.45	epoch-40-avg-20
0.25	2.54 & 5.78	2.72 & 5.45	2.72 & 5.45	2.66 & 5.29	2.65 & 5.29	epoch-40-avg-18
0.30	2.52 & 5.79	2.66 & 5.47	2.66 & 5.47	2.59 & 5.33	2.58 & 5.34	epoch-40-avg-14
0.40	2.5 & 5.89	2.69 & 5.57	2.69 & 5.57	2.63 & 5.46	2.63 & 5.52	epoch-40-avg-10