awd-lstm-lm
awd-lstm-lm copied to clipboard
AttributeError: 'Program' object has no attribute '_program'
I'm trying to reproduce QRNN in the project and launch the command from readme example:
(.venv3gpu) alx@x3:~/Cloud/spell_corr/awd-lstm-lm$ python -u main.py --epochs 14 --nlayers 4 --emsize 400 --nhid 2500 --alpha 0 --beta 0 --dropoute 0 --dropouth 0.1 --dropouti 0.1 --dropout 0.1 --wdrop 0 --wdecay 0 --bptt 140 --batch_size 60 --optimizer adam --lr 1e-3 --data data/wikitext-103 --save WT103.12hr.QRNN.pt --when 12 --model QRNN
Loading cached dataset...
Applying weight drop of 0.0 to weight
Applying weight drop of 0.0 to weight
Applying weight drop of 0.0 to weight
Applying weight drop of 0.0 to weight
[QRNNLayer(
(linear): WeightDrop(
(module): Linear(in_features=800, out_features=7500, bias=True)
)
), QRNNLayer(
(linear): WeightDrop(
(module): Linear(in_features=2500, out_features=7500, bias=True)
)
), QRNNLayer(
(linear): WeightDrop(
(module): Linear(in_features=2500, out_features=7500, bias=True)
)
), QRNNLayer(
(linear): WeightDrop(
(module): Linear(in_features=2500, out_features=1200, bias=True)
)
)]
Using [2800, 20000, 76000]
Args: Namespace(alpha=0.0, batch_size=60, beta=0.0, bptt=140, clip=0.25, cuda=True, data='data/wikitext-103', dropout=0.1, dropoute=0.0, dropouth=0.1, dropouti=0.1, emsize=400, epochs=14, log_interval=200, lr=0.001, model='QRNN', nhid=2500, nlayers=4, nonmono=5, optimizer='adam', resume='', save='WT103.12hr.QRNN.pt', seed=1111, tied=True, wdecay=0.0, wdrop=0.0, when=[12])
Model total parameters: 153886638
/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/torch/nn/functional.py:1340: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/torch/nn/functional.py:1351: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
Traceback (most recent call last):
File "main.py", line 240, in <module>
train()
File "main.py", line 196, in train
output, hidden, rnn_hs, dropped_rnn_hs = model(data, hidden, return_h=True)
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/alx/Cloud/spell_corr/awd-lstm-lm/model.py", line 81, in forward
raw_output, new_h = rnn(raw_output, hidden[l])
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/torchqrnn/qrnn.py", line 99, in forward
C = ForgetMult()(F, Z, hidden, use_cuda=self.use_cuda)
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/torchqrnn/forget_mult.py", line 179, in forward
return GPUForgetMult()(f, x, hidden_init) if use_cuda else CPUForgetMult()(f, x, hidden_init)
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/torchqrnn/forget_mult.py", line 120, in forward
self.compile()
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/torchqrnn/forget_mult.py", line 102, in compile
program = Program(kernel.encode(), 'recurrent_forget_mult.cu'.encode())
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/pynvrtc/compiler.py", line 52, in __init__
include_names)
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/pynvrtc/interface.py", line 200, in nvrtcCreateProgram
c_char_p(encode_str(src)), c_char_p(encode_str(name)),
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/pynvrtc/interface.py", line 54, in encode_str
return s.encode("utf-8")
AttributeError: 'bytes' object has no attribute 'encode'
Exception ignored in: <bound method Program.__del__ of <pynvrtc.compiler.Program object at 0x7fa9f5a4e518>>
Traceback (most recent call last):
File "/home/alx/Cloud/.venv3gpu/lib/python3.6/site-packages/pynvrtc/compiler.py", line 56, in __del__
self._interface.nvrtcDestroyProgram(self._program)
AttributeError: 'Program' object has no attribute '_program'
Does anybody know how to fix this issue?
I executed those two commands before the python -u main.py ....
command, seems to solve the issue
pip install cupy pynvrtc git+https://github.com/salesforce/pytorch-qrnn
pip install pynvrtc==8.0