For ASR, if I set num_cnn_layers = 0, Invalid argument: Matrix size-incompatible: In[0]: [96,1056], In[1]: [1120,4096]

[yujunlhz]

My setting is: ep = p.encoder ep.input_shape = [None, None, 96, 1] ep.lstm_cell_size = 1024 ep.num_lstm_layers = 4 ep.conv_filter_shapes = [] ep.conv_filter_strides = [] ep.num_conv_lstm_layers = 0 ep.num_cnn_layers = 0

The error message I got is:

2021-04-20 01:28:21.220217: I lingvo/core/ops/record_yielder.cc:614] Emitted 649 records from /tmp/lingvo/lingvo/speech_data/train/train.tfrecords-00044-of-00100 2021-04-20 01:28:21.221373: I lingvo/core/ops/record_yielder.cc:425] 0x7f8e689de720Basic record yielder exit I0420 01:28:21.910491 140268959409920 base_runner.py:120] Job trainer: Retrying as expected trainer exception: From /job:localhost/replica:0/task:0: 6 root error(s) found. (0) Invalid argument: Matrix size-incompatible: In[0]: [96,1056], In[1]: [1120,4096] [[{{node MatMul}}]] [[gradients/fprop/librispeech/tower_0_3/enc/proj_L2/MatMul_grad/MatMul_1_G1560]] (1) Invalid argument: Matrix size-incompatible: In[0]: [96,1056], In[1]: [1120,4096] [[{{node MatMul}}]] [[gradients/fprop/librispeech/tower_0_5/enc/brnn_L2/IdentityN_1_grad/While_G1600]] (2) Invalid argument: Matrix size-incompatible: In[0]: [96,1056], In[1]: [1120,4096] [[{{node MatMul}}]] [[gradients/fprop/librispeech/tower_0_3/dec/BiasAdd_grad/BiasAddGrad_G1228]] (3) Invalid argument: Matrix size-incompatible: In[0]: [96,1056], In[1]: [1120,4096] [[{{node MatMul}}]]

Any one can help?

[yujunlhz]

@jonathanasdf would you help with this issue? For ASR, if I set num_cnn_layers = 0, Invalid argument: Matrix size-incompatible: In[0]: [96,1056], In[1]: [1120,4096]

[jonathanasdf]

Hi, we are looking into it. Please give us some time.

[jonathanasdf]

Hi, we added a unit test in https://github.com/tensorflow/lingvo/commit/51724298228462ed3887c0ef605240bec3853ad4 and were unable to reproduce this.

Do you also get the same error with num_cnn_layers > 0? Please double check your input shapes.

[yujunlhz]

Ah, it should be ep.input_shape = [None, None, 32, 3]. I had thought that [None, None, 32, 3] can be change to [None, None, 96, 1] automatically. thanks

[yujunlhz]

Hey @jonathanasdf , if I wish to use MFCC feature instead fbank feature, I need to insert a few projection layers in front of the BLSTM layers in stead of CNN layers. I modify the encoder.py as following:

conv_output_shape = p.input_shape

params_proj_layers = []

proj_p = p.proj_tpl.Copy() proj_p.input_dim = conv_output_shape[2] proj_p.output_dim = 1024 proj_p.name = 'proj_L_pre_input' params_proj_layers.append(proj_p)

conv_output_shape[2]=1024

Seems not work: (0) Invalid argument: Matrix size-incompatible: In[0]: [96,608], In[1]: [1536,2048] [[{{node MatMul}}]] [[fprop/librispeech/tower_0_5/enc/brnn_L0/StatefulPartitionedCall/While]] (1) Invalid argument: Matrix size-incompatible: In[0]: [96,608], In[1]: [1536,2048] [[{{node MatMul}}]] [[fprop/librispeech/tower_0_3/enc/brnn_L0/StatefulPartitionedCall/While]] (2) Invalid argument: Matrix size-incompatible: In[0]: [96,608], In[1]: [1536,2048] [[{{node MatMul}}]] [[gradients/fprop/librispeech/tower_0_5/dec/IdentityN_grad/While_G712]] (3) Invalid argument: Matrix size-incompatible: In[0]: [96,608], In[1]: [1536,2048] [[{{node MatMul}}]]

Any suggestions for this configuration? Thanks

[jonathanasdf]

Hi, we cannot offer detailed debugging help to all users. Please try to debug it as much as you can yourself.

If you check FProp(), self.proj[i].FProp() happens after the rnn layers, while you are trying to project the input before rnn. So you cannot add to params_proj_layers, you have to do a separate CreateChild('input_proj', input_proj_params) and modify FProp() accordingly.