warp-rnnt

warp-rnnt copied to clipboard

Strange behavior using PyTorch DDP

7

@1ytic Hi, So far I have been able to use the loss with DDP on a single GPU , it behaves more or less as expected. But when I use...

snakers4

WARNING: sample 0 [42, 26] has a forward/backward mismatch -52.543503 / 0.000000

3

  what does the mismatch mean? my_env. python=3.7.0. torch=1.6.0. cuda=10.2.

songtaoshi

rnnt_loss status 1

2

I got the error which is shown as follows. python=3.8 torch verison=1.10.2 cudatoolkit=10.2.89 CUDA version=10.2. The GCC version is 5.4.0 

jiay7

question about the rnnt loss arguments

4

```python log_probs (torch.FloatTensor): Input tensor with shape (N, T, U, V) where N is the minibatch size, T is the maximum number of input frames, U is the maximum number...

songtaoshi

undefined symbol: _ZNSt19basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev

2

/***/warp-rnnt/pytorch_binding/warp_rnnt/_C.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZNSt19basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev

wylqq312715289

warp-rnnt with compact memory layout implementation

11

The **compact layout** in memory can be explained with this figure. The input to `rnnt_loss()` is of size `(N, T, U+1, V)=(3, 6, 6, V)` in a normal layout. Colored...

maxwellzh

Question about average_frames and reduction parmas

1

I want to have a stable loss which is rubust to labels_lengths when training. What value should I pass to this two parmas? What's more, what is the approximate relationship...

wl-junlin

warning that forward/backward mismatch

3

The warning messages occasionally thrown out during training, ``` ... WARNING: sample 10 [81, 25] has a forward/backward mismatch -0.000083 / -0.000083 ... WARNING: sample 11 [62, 28] has a...

maxwellzh

Transducer loss leads to memory leak

4

Hi, I'm using rnnt-loss and pytorch-lightning to train my model. But I found the 4D tensor which is used to calculate transducer will be accumulated in GPU, when I check...

YangHao97

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed。

1

I search the error in the title,it happens when there are several losses.But in my code,there is only the RNN-t-loss,but it gives the error. The full sentence is ''RuntimeError: Trying...

lfgogogo