Rudolf A. Braun

wav2vec2 pretraining implemented with speechbrain

20

(this is not meant for merge yet but to share the code) This allows one to pretrain a wav2vec2 model without relying on fairseq or huggingface. It follows the fairseq...

Brain's check_gradients seems to not do what name implies?

2

Was trying to debug something of mine and noticed two things: 1. [check_gradients](https://github.com/speechbrain/speechbrain/blob/develop/speechbrain/core.py#L888) does not actually check the gradients, it checks the loss [which does not have to be NaN...

Value of silence_weight

7

Hi Nickolay! I've started using the vosk-api, and I've had a few utterances where the output was worse compared to when doing decoding with kaldi's tcp binary (stuff like no...

Loading audiofiles with offset is broken since 0.10

4

### 🚀 The feature ## Describe Had to downgrade to 0.9 so that it would work. Tried nightly and 0.10 ``` >>> import torchaudio >>> torchaudio.load('/path/to/wav/5ccae615b4e948578998a20f-wav.wav', frame_offset=10351280, num_frames=67232) trim: Error...

Trying to use 2 GPUs results in neverending process that can't be killed without restart

11

I have two 1080tis. When I try and use kmcuda with both by setting CUDA_VISIBLE_DEVICES to the GPUs I use for compute, (I've tried this out with device=0 and device=3),...

Sometimes batches are created which do not have same number of supervisions and inputs

2

``` Traceback (most recent call last): File "train.py", line 1019, in main() File "train.py", line 1012, in main run(rank=0, world_size=1, args=args) File "train.py", line 867, in run scan_pessimistic_batches_for_oom( File "train.py",...

After training if I write out the vocabulary: ``` for w in bpe.vocab(): fh.write(f'{w}\n') // fh is filehandler ``` and then look inside the file this is (a subset) of...

Removing newlines and punctuation from the input

1

Had to add the `regex` package as `re` does not have a punctuation class.

Bizarre predictions

3

Think the results below speak for themselves. It would be helpful to know what sort of normalisation is expected since it seems WTL does not do any. Mostly seems to...

Doesn't this loss function have the issue that the beginning time steps will get a much larger gradient than the final ones?

1

I want to confirm that the issues I'm experiencing are a fundamental issue with the loss and not my implementation (which is a slight modification of this). It seems to...