RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment

[Sreyan88]

Hi there!

Great repo! I am trying to pre-train wav2vec2 using this and I get the error:

RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment

any fix for this?

[arxyzan]

Hi Sreyan, thanks for your feedback. This issue is caused by the recent commit on EMA. It seems that some PyTorch modules cannot be deepcopied. I'm working on the problem and trying to fix it. Stay tuned!

Best, Aryan

[Sreyan88]

Hi @arxyzan ,

Thank you for the reply. I also had the question if the code worked (till backprop while pre-training from your end). I made some hacks for the above loss, however, there seem to be some more errors:

This line here seems to produce an output of (batch_size x time_steps x time_steps x 768) for audio pre-training which is not expected. This in turn is returning an error while loss calculation.

Can you please check into these? Any help would be appreciated! Thanks much!

[arxyzan]

Can you paste the output of your error? I'm pretty sure it worked the last time I ran it.

[Sreyan88]

File "train.py", line 25, in <module>
    trainer.train()
  File "/speech/sreyan/slt/data2vec-pytorch/audio/trainer.py", line 140, in train
    train_loss = self.train_epoch(epoch)
  File "/speech/sreyan/slt/data2vec-pytorch/audio/trainer.py", line 105, in train_epoch
    loss = self.train_step(batch)
  File "/speech/sreyan/slt/data2vec-pytorch/audio/trainer.py", line 66, in train_step
    loss.backward()
  File "/speech/sreyan/anaconda3/envs/data2vec/lib/python3.7/site-packages/torch/_tensor.py", line 307, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/speech/sreyan/anaconda3/envs/data2vec/lib/python3.7/site-packages/torch/autograd/__init__.py", line 150, in backward
    grad_tensors_ = _make_grads(tensors, grad_tensors_)
  File "/speech/sreyan/anaconda3/envs/data2vec/lib/python3.7/site-packages/torch/autograd/__init__.py", line 51, in _make_grads
    raise RuntimeError("grad can be implicitly created only for scalar outputs")
RuntimeError: grad can be implicitly created only for scalar outputs

[Sreyan88]

You can try running :

python train.py --config audio/configs/wav2vec2-pretraining.yaml

to reproduce.

[arxyzan]

Unfortunately, I don't have access to the datasets anymore, hence cannot run pretraining script right now. Instead can you get the shape of loss tensor before loss.backward()?

[Sreyan88]

Hi, Sorry for the delay, I will post it today, was busy with conference deadlines. @arxyzan can you please re-open this?