deep3d icon indicating copy to clipboard operation
deep3d copied to clipboard

error when running the ipynb file.

Open fivejjs opened this issue 9 years ago • 15 comments

if not os.path.exists('deep3d-0050.params'):
    urllib.urlretrieve('http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params', 'deep3d-0050.params')
model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))

The error gives:

---------------------------------------------------------------------------
MXNetError                                Traceback (most recent call last)
<ipython-input-7-9deaeed89c01> in <module>()
      1 if not os.path.exists('deep3d-0050.params'):
      2     urllib.urlretrieve('http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params', 'deep3d-0050.params')
----> 3 model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.pyc in load(prefix, epoch, ctx, **kwargs)
    832         - ``prefix-epoch.params`` will be saved for parameters.
    833         """
--> 834         symbol, arg_params, aux_params = load_checkpoint(prefix, epoch)
    835         return FeedForward(symbol, ctx=ctx,
    836                            arg_params=arg_params, aux_params=aux_params,

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.pyc in load_checkpoint(prefix, epoch)
    361     - parameters will be loaded from ``prefix-epoch.params``.
    362     """
--> 363     symbol = sym.load('%s-symbol.json' % prefix)
    364     save_dict = nd.load('%s-%04d.params' % (prefix, epoch))
    365     arg_params = {}

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/symbol.pyc in load(fname)
    932         raise TypeError('fname need to be string')
    933     handle = SymbolHandle()
--> 934     check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
    935     return Symbol(handle)
    936 

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/base.pyc in check_call(ret)
     75     """
     76     if ret != 0:
---> 77         raise MXNetError(py_str(_LIB.MXGetLastError()))
     78 
     79 def c_str(string):

MXNetError: Failed loading Op softmax of type SoftmaxActivation: Cannot find argument 'type', Possible Arguments:
----------------
mode : {'channel', 'instance'},optional, default='instance'
    Softmax Mode. If set to instance, this operator will compute a softmax for each instance in the batch; this is the default mode. If set to channel, this operator will compute a num_channel-class softmax at each position of each instance; this can be used for fully convolutional network, image segmentation, etc.```

It looks like:
```sym = symbol.load('deep3d-symbol.json')

How to fix it? Thanks

fivejjs avatar May 14 '16 08:05 fivejjs

@fivejjs I have met this problem too. It is because that the "MXNET" you download is the latest version, and the version the author use is a previous version.

In the latest version, when u want to use SoftmaxActivation, u should replace the "type" into "mode" in the json file. But after u do this, u will meet another CudnnBatchNorm problem. Then u should change all "CudnnBatchNorm" to "BatchNorm" in the json file, and modify the model.aux_params by replace "moving_inv_var" into "moving_var". Then it will be all ok to run.

The other easy solution is u download a previous version of MXNET for example "tag 0321"

Feng137 avatar May 17 '16 06:05 Feng137

@fivejjs However, if you compile mxnet using Cuda7.0 and cudnn 4 or >, then you shouldn't have the cudnn problem either, even using the latest mxnet.

@azrael0fog have you managed to run the convert_movie.py? mine runs until it hits line 285 in data.py and complaining of Attribute Error: 'NoneType' object has no attribute split. As if there is no data in the list.

lasandcris avatar May 23 '16 11:05 lasandcris

Couple questions--where can I find model.aux_params? And where do I get the .ctx file that convert_movie.py needs as an argument?

I'm encountering this error too...Ubuntu 14.04 with CUDA 7 / CUDNN 4 and CUDA 7.5 / CUDNN 5. Tried search-and-replace in json file, building from tag 0321, no luck.

n1ckfg avatar Jun 01 '16 16:06 n1ckfg

i am having the same issue... Ubuntu 16.04 CUDA 7.5 / CUDNN 4 and i keep getting the same error:

ValueError: Find name bn_pool3_moving_inv_var that is not in the auxiliary states

i tried changing the deep3d-0050.params file from moving_inv_var to moving_var. Then the model doesnot load. Can you specify the exact location to change the moving_inv_var file.

dineshreddy91 avatar Jun 06 '16 14:06 dineshreddy91

@dineshreddy91 I don't know whether the CUDA 7.5 and CUDNN 4 is match. When using CUDA 7.5, I use CUDNN 5; when using CUDA 7, I use CUDNN 4. Both works.

Though both works, when using CUDA 7.5/CUDNN 5, the result picture is not the same as the author shows. I guess the author is using CUDA 7/CUDNN 4.

I am not sure, but I guess there may some difference of CUDA 7.5 and CUDA 7 when calculating.

Highly recommanded changing the version of CUDA into version7.

Feng137 avatar Jun 07 '16 04:06 Feng137

@azrael0fog I am not getting how are you modifying model.aux_params. Please can you explain it. Where is the file and how to modify it.

ashishgupta-skg avatar Aug 02 '16 06:08 ashishgupta-skg

Still no fix?

fivejjs avatar Aug 02 '16 13:08 fivejjs

Can you suggest any way to modify deep3d-0050.params. I am really struggling with it.

ashishgupta-skg avatar Aug 02 '16 14:08 ashishgupta-skg

I somewhat understand his meaning, model.aux_params is not a file, it's a variable, the modification should be done in the program.

Greenleaf88 avatar Aug 04 '16 11:08 Greenleaf88

@azrael0fog I have done all those modifications: In the latest version, when u want to use SoftmaxActivation, u should replace the "type" into "mode" in the json file. But after u do this, u will meet another CudnnBatchNorm problem. Then u should change all "CudnnBatchNorm" to "BatchNorm" in the json file, and modify the model.aux_params by replace "moving_inv_var" into "moving_var". Then it will be all ok to run.

But still got this error: mxnet.base.MXNetError: [19:49:17] src/operator/cudnn_batch_norm.cc:20: CuDNNBatchNorm is merged into BatchNorm for cudnn version above v5.Use the later instead.

Any advise?

Greenleaf88 avatar Aug 04 '16 11:08 Greenleaf88

degrade the cudnn version from v5 to v4, it works.

Greenleaf88 avatar Aug 05 '16 07:08 Greenleaf88

@azrael0fog I am a new comer to mxnet .I want to use deep3d to make a few pictures. I want to know which file need to modify to aviod the CudnnBatchNorm problem,.I got in the trouble for a few days. I need your help Thanks a lot.

gudongxing avatar Oct 07 '16 12:10 gudongxing

The solution by @azrael0fog worked in my case. I am using mxnet(v0.9.3) and CUDA 8.0 with cudnn downgraded to v4. Modify line 962 in "deep3d-symbol.json", which I believe defines the structure of the network, as following: FROM

{
      "op": "SoftmaxActivation", 
      "param": {"type": "channel"}, 
      "name": "softmax", 
      "inputs": [[105, 0]], 
      "backward_source_id": -1
    }

TO

{
      "op": "SoftmaxActivation", 
      "param": {"mode": "channel"}, 
      "name": "softmax", 
      "inputs": [[105, 0]], 
      "backward_source_id": -1
    }

However, the difference with @azrael0fog's case is that I did not have to change "CudnnBatchNorm" and "model.aux_params" at all. Hope this works in yours.

WaratenZ avatar Feb 16 '17 13:02 WaratenZ

@Greenleaf88 , do you know which file(s) need to change model.aux_params? I didn't find any file include model.aux_params. If possible, can you list the files' name. Thanks.

yishi2013 avatar Feb 24 '17 20:02 yishi2013

I find how to change model.aux_params from https://github.com/piiswrong/deep3d/issues/23. Thanks @zedomel

model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))

model.aux_params['bn_pool3_moving_var'] = model.aux_params['bn_pool3_moving_inv_var'] model.aux_params['bn_pool2_moving_var'] = model.aux_params['bn_pool2_moving_inv_var'] model.aux_params['bn_pool4_moving_var'] = model.aux_params['bn_pool4_moving_inv_var'] model.aux_params['bn_pool1_moving_var'] = model.aux_params['bn_pool1_moving_inv_var'] del model.aux_params['bn_pool1_moving_inv_var'] del model.aux_params['bn_pool2_moving_inv_var'] del model.aux_params['bn_pool3_moving_inv_var'] del model.aux_params['bn_pool4_moving_inv_var']

yishi2013 avatar Feb 24 '17 20:02 yishi2013