DeepLearningExamples icon indicating copy to clipboard operation
DeepLearningExamples copied to clipboard

[SpeechSynthesis/Tactron2] Training crashes with AttributeError: module 'torch._C' has no attribute '_jit_set_autocast_mode'

Open mikesol opened this issue 2 years ago • 1 comments

Related to Model/Framework(s) https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/SpeechSynthesis/Tacotron2

Describe the bug

The model cannot train because it hits the following error once in the docker image:

root@eff550f3bccb:/workspace/tacotron2# bash scripts/train_waveglow.sh
Traceback (most recent call last):
  File "train.py", line 42, in <module>
    import models
  File "/workspace/tacotron2/models.py", line 33, in <module>
    from waveglow.model import WaveGlow
  File "/workspace/tacotron2/waveglow/model.py", line 28, in <module>
    torch._C._jit_set_autocast_mode(False)
AttributeError: module 'torch._C' has no attribute '_jit_set_autocast_mode'
Traceback (most recent call last):
  File "train.py", line 42, in <module>
    import models
  File "/workspace/tacotron2/models.py", line 33, in <module>
    from waveglow.model import WaveGlow
  File "/workspace/tacotron2/waveglow/model.py", line 28, in <module>
    torch._C._jit_set_autocast_mode(False)
AttributeError: module 'torch._C' has no attribute '_jit_set_autocast_mode'
root@eff550f3bccb:/workspace/tacotron2# exit

To Reproduce Steps to reproduce the behavior:

  1. Follow the readme instructions verbatim for training waveglow. The failure happens on bash scripts/train_waveglow.sh.

Environment Please provide at least:

  • Container version (e.g. pytorch:19.05-py3): NVIDIA Release 20.06 (build 13419386), PyTorch Version 1.6.0a0+9907a3e
  • GPUs in the system: 2x 1080TI
  • CUDA driver version (e.g. 418.67): Cuda compilation tools, release 11.6, V11.6.124, Build cuda_11.6.r11.6/compiler.31057947_0

mikesol avatar Sep 12 '23 06:09 mikesol

same error

sp1cae avatar Jul 16 '24 05:07 sp1cae