Tatiana Likhomanenko

What is your maximum target and input length in the dataset? Is it crashing on the same GPU or can happen on different ones? cc @avidov @jacobkahn

Could you try to run training on 7 GPUs excluding the 4th one, just to make sure it is the problem with the 4th GPU.

Data input varies in the input size. It could be large input/target sizes in the batch which cause this. Can you plot stats on input size and target size in...

Am I understand that it missing words with upper-case? or do you have all lower-cased words?

Could you post head of your tokens set and lexicon?

Yep, so at first you can check Viterbi only with Test to see if the acoustic model itself have this problem. One thing is you are using lexicon-based decoder, but...

The error is `nvcc fatal : Unsupported gpu architecture 'compute_30'`. So you have very recent cuda version, probably we didn't have the fix for it in the version you are...

Feel free to reopen if this suggestion didn't resolve the issue for you (for flashlight version of python bindings please create issue directly in the flashlight git).

what is GPU card?

cc @avidov