Piotr Żelasko

I moved the discussion to Lhotse issue https://github.com/lhotse-speech/lhotse/issues/785#issuecomment-1262978092

Is it possible that you have slow disks/NFS? Reading audio possibly requires much more I/O than reading features so you might get bottlenecked by this. On the grid where I...

> Do you think you can get full utilization out of 2 or 4 GPUs as well? On yet another grid (a good one) I was training GigaSpeech model with...

I didn't run into it but somebody else did recently. There's a [WIP PR in Lhotse](https://github.com/lhotse-speech/lhotse/pull/595), I'll either ask Ondra if he can to finish it or maybe I will.

> What will be the simplest way to extract features that mimic one epoch worth of features in advance mimicking the recipe in asr_datamodule of fisher recipe? I am thinking...

After the changes it seems to work (sometimes). In general it seems to consume much more GPU memory, I have been decreasing max_frames and the beam for intersect_dense for dens,...

Hmm, I think the latter error was related to having too few outputs in the nnet (I was off by two). I fixed that and the error disappeared...

FYI I updated to the most recent K2 because I remembered there were some new memory optimizations for intersection; it does help. For dens intersection with posteriors, I am now...

I am getting an error during decoding graph composition, @csukuangfj @qindazhu @danpovey can you suggest what would be the right approach to debugging it (or what could be the cause)?...

BTW the training seems to have gone OK ``` 2021-03-16 05:41:50,279 INFO [mmi_att_transformer_train.py:336] Validation average objf: 0.151131 over 481977.0 frames (100.0% kept) 2021-03-16 05:42:09,358 INFO [mmi_att_transformer_train.py:311] batch 3610, epoch 9/10...