pytorch->IR: AttributeError: Can't get attribute 'Net' on <module '__main__' from '/usr/local/bin/mmtoir'>

[Ruochen0715]

trafficstars

I use Pytorch 0.4.0 and I can convert resnet101 to IR. However, when I tried to transform my model to IR, I encountered a problem. For example, the following models are defined and trained.

class Net(nn.Module):

def __init__(self):
    super(Net, self).__init__()
    self.conv1 = nn.Conv2d(3, 6, 5)
    self.pool = nn.MaxPool2d(2, 2)
    self.conv2 = nn.Conv2d(6, 16, 5)
    self.fc1 = nn.Linear(16 * 5 * 5, 120)
    self.fc2 = nn.Linear(120, 84)
    self.fc3 = nn.Linear(84, 10)

def forward(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    x = x.view(-1, 16 * 5 * 5)
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)
    return x

Next, I saved the network structure and parameters at the same time: net = Net() PATH = './cifar_whole.pth' torch.save(net, PATH)

However, when I tried to transform the model with the following command, an error occurred: mmtoir -f pytorch -d cifar_whole --inputShape 3,32,32 -n cifar_whole.pth

The error is: File "/usr/local/bin/mmtoir", line 11, in sys.exit(_main()) File "/usr/local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 192, in _main ret = _convert(args) File "/usr/local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 92, in _convert parser = PytorchParser(model, inputshape[0]) File "/usr/local/lib/python3.6/site-packages/mmdnn/conversion/pytorch/pytorch_parser.py", line 78, in init model = torch.load(model_file_name, map_location='cpu') File "/root/.local/lib/python3.6/site-packages/torch/serialization.py", line 303, in load return _load(f, map_location, pickle_module) File "/root/.local/lib/python3.6/site-packages/torch/serialization.py", line 469, in _load result = unpickler.load() AttributeError: Can't get attribute 'Net' on <module 'main' from '/usr/local/bin/mmtoir'>

It seems that even if the parameters and network structure are saved at the same time, the class definition of the network is still needed for torch to read the pth file. When I convert resnet101 to IR, I don't encounter this problem, because 'torchvision.models.resnet.ResNet' has the definition of this network.

I also saw the extractor in Example: https://github.com/Microsoft/MMdnn/blob/master/mmdnn/conversion/examples/pytorch/extractor.py#L50 But this extractor only contains "download" and "inference" functions. It seems that it cannot output IR files.

So, how can I transform my own defined network structure to IR through mmdnn? thank you very much

[rainLiuplus]

Hi @Ruochen0715, you could paste your model definition on the /usr/local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py to have a try. Thanks

[Ruochen0715]

Hi @rainLiuplus , Please help me T^T

I added the following codes at the beginning of “/usr/local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py”

//=================================================== import sys as _sys import google.protobuf.text_format as text_format from six import text_type as _text_type

import torch.nn as nn class Net(nn.Module):

def __init__(self):
    super(Net, self).__init__()
    self.conv1 = nn.Conv2d(3, 6, 5)
    self.pool = nn.MaxPool2d(2, 2)
    self.conv2 = nn.Conv2d(6, 16, 5)
    self.fc1 = nn.Linear(16 * 5 * 5, 120)
    self.fc2 = nn.Linear(120, 84)
    self.fc3 = nn.Linear(84, 10)

def forward(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    x = x.view(-1, 16 * 5 * 5)
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)
    return x

print("The Net has been defined!!!!!!!!!!")

def _convert(args): and so on.... //============================================

But nothing happend.. The result is

root@07b9420edadf:/dfsdata2/wangrc2_data/TryTorch# mmtoir -f pytorch -d cifar_cpu_whole --inputShape 3,32,32 -n cifar_cpu_whole.pth The Net has been defined!!!!!!!!!! Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/mmdnn/conversion/pytorch/pytorch_parser.py", line 76, in init model = torch.load(model_file_name) File "/root/.local/lib/python3.6/site-packages/torch/serialization.py", line 302, in load return _load(f, map_location, pickle_module) File "/root/.local/lib/python3.6/site-packages/torch/serialization.py", line 468, in _load result = unpickler.load() AttributeError: Can't get attribute 'Net' on <module 'main' from '/usr/local/bin/mmtoir'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/mmtoir", line 11, in sys.exit(_main()) File "/usr/local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 212, in _main ret = _convert(args) File "/usr/local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 112, in _convert parser = PytorchParser(model, inputshape[0]) File "/usr/local/lib/python3.6/site-packages/mmdnn/conversion/pytorch/pytorch_parser.py", line 78, in init model = torch.load(model_file_name, map_location='cpu') File "/root/.local/lib/python3.6/site-packages/torch/serialization.py", line 302, in load return _load(f, map_location, pickle_module) File "/root/.local/lib/python3.6/site-packages/torch/serialization.py", line 468, in _load result = unpickler.load() AttributeError: Can't get attribute 'Net' on <module 'main' from '/usr/local/bin/mmtoir'>

I can see "The Net has been defined!!!!!!!!!!" in the result, so I did change the program, but useless.......

Please help me T_T...........

[Ruochen0715]

@rainLiuplus My dear DaLao, Please Dai Dai me T^T

[JennyVVV]

I have the totally same problem, did you solve this problem?

[buildist]

After modifying convertToIR.py, try the command python /usr/local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py -f pytorch -d cifar_cpu_whole --inputShape 3,32,32 -n cifar_cpu_whole.pth instead of mmtoir -f pytorch -d cifar_cpu_whole --inputShape 3,32,32 -n cifar_cpu_whole.pth

[egiacomazzi]

I also have the same problem. I'm running the commands in the docker container. When executing this python /usr/local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py -f pytorch -d cifar_cpu_whole --inputShape 3,32,32 -n cifar_cpu_whole.pth command the following error is thrown:

Traceback (most recent call File "/usr/local/lib/python3.5/dist-packages/mmdnn/conversion/_script/convertToIR.py", line 2, in import google.protobuf.text_format as text_format ImportError: No module named google.protobuf.text_format

And with this command mmtoir -f pytorch -d cifar_cpu_whole --inputShape 3,32,32 -n cifar_cpu_whole.pth the error looks like this:

Traceback (most recent call last): File "/usr/local/bin/mmtoir", line 11, in sys.exit(_main()) File "/usr/local/lib/python3.5/dist-packages/mmdnn/conversion/_script/convertToIR.py", line 292, in _main ret = _convert(args) File "/usr/local/lib/python3.5/dist-packages/mmdnn/conversion/_script/convertToIR.py", line 187, in _convert parser = PytorchParser(args.network, inputshape) File "/usr/local/lib/python3.5/dist-packages/mmdnn/conversion/pytorch/pytorch_parser.py", line 71, in init model = torch.load(model_file_name) File "/usr/local/lib/python3.5/dist-packages/torch/serialization.py", line 303, in load return _load(f, map_location, pickle_module) File "/usr/local/lib/python3.5/dist-packages/torch/serialization.py", line 459, in _load magic_number = pickle_module.load(f) _pickle.UnpicklingError: A load persistent id instruction was encountered, but no persistent_load function was specified.

The second error looks a lot like what is mentioned here but when I try to build the model (add the structure to the weights) in the docker container it shows that the torch module can not be found.

Can anyone help me with this problem?

[KerwinKai]

I also have the same problem. I'm running the commands in the docker container. When executing this python /usr/local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py -f pytorch -d cifar_cpu_whole --inputShape 3,32,32 -n cifar_cpu_whole.pth command the following error is thrown:

Traceback (most recent call File "/usr/local/lib/python3.5/dist-packages/mmdnn/conversion/_script/convertToIR.py", line 2, in import google.protobuf.text_format as text_format ImportError: No module named google.protobuf.text_format

And with this command mmtoir -f pytorch -d cifar_cpu_whole --inputShape 3,32,32 -n cifar_cpu_whole.pth the error looks like this:

Traceback (most recent call last): File "/usr/local/bin/mmtoir", line 11, in sys.exit(_main()) File "/usr/local/lib/python3.5/dist-packages/mmdnn/conversion/_script/convertToIR.py", line 292, in _main ret = _convert(args) File "/usr/local/lib/python3.5/dist-packages/mmdnn/conversion/_script/convertToIR.py", line 187, in _convert parser = PytorchParser(args.network, inputshape) File "/usr/local/lib/python3.5/dist-packages/mmdnn/conversion/pytorch/pytorch_parser.py", line 71, in init model = torch.load(model_file_name) File "/usr/local/lib/python3.5/dist-packages/torch/serialization.py", line 303, in load return _load(f, map_location, pickle_module) File "/usr/local/lib/python3.5/dist-packages/torch/serialization.py", line 459, in _load magic_number = pickle_module.load(f) _pickle.UnpicklingError: A load persistent id instruction was encountered, but no persistent_load function was specified.

The second error looks a lot like what is mentioned here but when I try to build the model (add the structure to the weights) in the docker container it shows that the torch module can not be found.

Can anyone help me with this problem?

I also encountered this problem. Do you have a solution to solve it.