deep-visualization-toolbox

deep-visualization-toolbox copied to clipboard

Hello, First of all thank you for your fantastic job in creating the toolbox. second of all I would like to address several matters that confused me. As far as I know, the latest version of caffe supports deconvolutional layers as well, knowing that and also this that, the readme was created for the former version initially, and seemingly has not been updated so far:

1. Is it still mandatory to install the specific version of caffe mentioned in readme step 1(deconv-deep-vis-toolbox branch of caffe)?
2. Is there some specific reasons, as to why this toolbox is not said to as being compatible with windows? since caffe is officially supported now, and I guess the rest can be easily ported to windows. this made me before going and doing anything, ask whether the amount of changes the needed an update in the guide is more than just these issues I noticed.

Thanks alot again

by the way,I successfully ran this on windows 10 x64 ;). Everything seems to be working except that when there is no devonv layer, and you try to use the a key(or accidentally hit the key), you get AttributeError: 'Classifier' object has no attribute 'backward_from_layer' error and then nothing works except iterating next/previous static file (e.i image), and you need to restart the application .

check this it is not windows specific issue.

https://github.com/yosinski/deep-visualization-toolbox/issues/36

@Coderx7 I've made substantial work to the tool, but the tool still requires changes not in caffe master I've rebased from the master the caffe used, you can find it here https://github.com/arikpoz/caffe

Also note that caffe is now a submodule of the tool so installing the tool is as simple as:

git clone --recursive https://github.com/arikpoz/deep-visualization-toolbox.git  
cd deep-visualization-toolbox && ./build_default.sh

Latest version is in https://github.com/arikpoz/deep-visualization-toolbox

@arikpoz : Thanks, what are the new changes?

@Coderx7, I've added new features, made the tool generic and fixed countless bugs. There are over 100+ squashed commits, I've listed the new features in this PR https://github.com/yosinski/deep-visualization-toolbox/pull/128

@arikpoz , I think you really should decouple caffe from the toolbox itself, this makes it much easier to use on other platforms such as windows. and the settings have become very confusing. I would prefer the old style though.

@arikpoz : tried to run the new version, but faced with these errors? any idea whats going on here?

G:\Yosinski\deep-visualization-toolbox>run_toolbox.py
got module <module 'caffevis.app' from 'G:\Yosinski\deep-visualization-toolbox\caffevis\app.pyc'>
got app <class 'caffevis.app.CaffeVisApp'>
Got settings <module 'settings' from 'G:\Yosinski\deep-visualization-toolbox\settings.pyc'>
C:\Users\Master\Anaconda2\lib\site-packages\caffe\pycaffe.py:13: RuntimeWarning: to-Python converter for class boost::shared_ptr<class caffe::Net<float> > already registered; second conversion method ignored.
 from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
C:\Users\Master\Anaconda2\lib\site-packages\caffe\pycaffe.py:13: RuntimeWarning: to-Python converter for class boost::shared_ptr<class caffe::Blob<float> > already registered; second conversion method ignored.
 from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
C:\Users\Master\Anaconda2\lib\site-packages\caffe\pycaffe.py:13: RuntimeWarning: to-Python converter for class boost::shared_ptr<class caffe::Solver<float> > already registered; second conversion method ignored.
 from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1017 08:29:05.730690  1080 common.cpp:36] System entropy source not available, using fallback algorithm to generate seed instead.
Loaded caffe in CPU mode
The system cannot find the path specified.
Traceback (most recent call last):
 File "G:\Yosinski\deep-visualization-toolbox\run_toolbox.py", line 34, in <module>
   main()
 File "G:\Yosinski\deep-visualization-toolbox\run_toolbox.py", line 24, in main
   lv = LiveVis(settings)
 File "G:\Yosinski\deep-visualization-toolbox\live_vis.py", line 61, in __init__
   app = app_class(settings, self.bindings)
 File "G:\Yosinski\deep-visualization-toolbox\caffevis\app.py", line 48, in __init__
   self.net, self._data_mean = load_network(settings)
 File "G:\Yosinski\deep-visualization-toolbox\settings_misc.py", line 120, in load_network
   deduce_calculated_settings_without_network(settings)
 File "G:\Yosinski\deep-visualization-toolbox\settings_misc.py", line 15, in deduce_calculated_settings_without_network
   read_network_dag(settings)
 File "G:\Yosinski\deep-visualization-toolbox\settings_misc.py", line 228, in read_network_dag
   for parent_layer_name in top_to_layers[bottom]:
KeyError: u'data'

top_to_layers['data'] fails which means you don't have any layer which has a top set to 'data', but you do have a layer which depends on this blob..

Can you post your prototxt, or at least the start of it, the parts relevant to data definition?

@arikpoz : The irony is it works perfectly fine in Linux (actually in the Windows Subsystem for Linux (WSL) which is essentially a xenial ubuntu (16.04) ) here is the sample prototxt:

name: "CIFAR10_13"
input: "data"
input_dim: 1
input_dim: 3
input_dim: 32
input_dim: 32
force_backward: true
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  param {
    lr_mult: 1
  }
  convolution_param {
    num_output: 66
    kernel_size: 3
	pad:1
    stride: 1
    bias_term: true
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "bn1"
  type: "BatchNorm"
  bottom: "conv1"
  top: "conv1"
  param {
    lr_mult: 0
	 decay_mult: 0
  }
  param {
    lr_mult: 0
	 decay_mult: 0
  }
  param {
    lr_mult: 0
	 decay_mult: 0
  }
    include {
    phase: TRAIN
  }
    batch_norm_param {
    use_global_stats: false
    moving_average_fraction: 0.95
  }
}
layer {
  name: "bn1"
  type: "BatchNorm"
  bottom: "conv1"
  top: "conv1"
  param {
    lr_mult: 0
	 decay_mult: 0
  }
  param {
    lr_mult: 0
	 decay_mult: 0
  }
  param {
    lr_mult: 0
	 decay_mult: 0
  }
    include {
    phase: TEST
  }
    batch_norm_param {
    use_global_stats: true
    moving_average_fraction: 0.95
  }
}
layer {
  name: "scale1"
  type: "Scale"
  bottom: "conv1"
  top: "conv1"
  scale_param {
    bias_term: true
  }
}
layer {
  name: "relu1"
  type: "ReLU"
  bottom: "conv1"
  top: "conv1"
}
# #######DropOut##########
layer {
 name: "drop1"
 type: "Dropout"
 bottom: "conv1"
 top: "conv1"
 dropout_param {
  dropout_ratio: 0.2
  }
 }
#########################
layer {
  name: "conv1_0"
  type: "Convolution"
  bottom: "conv1"
  top: "conv1_0"
  param {
    lr_mult: 1
  }
  convolution_param {
    num_output: 128
    pad: 1
    kernel_size: 3
    stride: 1
    bias_term: true
    weight_filler {
      type: "xavier"
    }
  }
}
layer {
  name: "bn1_0"
  type: "BatchNorm"
  bottom: "conv1_0"
  top: "conv1_0"
  param {
    lr_mult: 0
	 decay_mult: 0
  }
  param {
    lr_mult: 0
	 decay_mult: 0
  }
  param {
    lr_mult: 0
	 decay_mult: 0
  }
    include {
    phase: TRAIN
  }
    batch_norm_param {
    use_global_stats: false
    moving_average_fraction: 0.95
  }
}
layer {
  name: "bn1_0"
  type: "BatchNorm"
  bottom: "conv1_0"
  top: "conv1_0"
  param {
    lr_mult: 0
	 decay_mult: 0
  }
  param {
    lr_mult: 0
	 decay_mult: 0
  }
  param {
    lr_mult: 0
	 decay_mult: 0
  }
    include {
    phase: TEST
  }
    batch_norm_param {
    use_global_stats: true
    moving_average_fraction: 0.95
  }
}
layer {
  name: "scale1_0"
  type: "Scale"
  bottom: "conv1_0"
  top: "conv1_0"
  scale_param {
    bias_term: true
  }
}
layer {
  name: "relu1_0"
  type: "ReLU"
  bottom: "conv1_0"
  top: "conv1_0"
}
######DropOut##########
layer {
 name: "drop2"
 type: "Dropout"
 bottom: "conv1_0"
 top: "conv1_0"
 dropout_param {
  dropout_ratio: 0.2
  }
 }
#######################
layer {
  name: "conv2"
  type: "Convolution"
  bottom: "conv1_0"
  top: "conv2"
  param {
    lr_mult: 1
  }
  convolution_param {
    num_output: 128
    kernel_size: 3
	pad:1
	stride: 1
    bias_term: true
    weight_filler {
      type: "gaussian"
      std: 0.01
    }
  }
}
...

it works perfectly fine in Linux

Can you explain what you mean by that? where does it work? where it doesn't?

It seems the problem is the use of old style caffe input. Try to replace:

input: "data"
input_dim: 1
input_dim: 3
input_dim: 32
input_dim: 32

with

layer {
  name: "input"
  type: "Input"
  top: "data"
  input_param {
    shape {
      dim: 1
      dim: 3
      dim: 32
      dim: 32
    }
  }
}

When the tool runs properly on Linux, the first thing it does is to run the caffe tool /build/tools/upgrade_net_proto_text.bin on your network file, creating a new prototxt in the latest caffe format. This allows the tool to only support the latest caffe syntax. For some reason this didn't happen with your file. Possibly because of the special host. If you ran it in Linux, you can see the converted prototxt file, and copy it to your other environment to use. If the original file name was deploy.prototxt then the new one would be deploy.prototxt.processed_by_deepvis

I'm on Windows 10 x64. I have also enabled windows 10s Linux Subsytem which lets me install and run linux based apps. I followed the instructions you gave earlier on building and using the toolbox and everything went fine. I tried using it on windows itself (not using linux subsystem obviously) and faced the issue I posted. You were right, by changing the data layer syntax that error is gone, however a new one poped up!:

Loaded mean from numpy file, data_mean.shape:  (3L, 32L, 32L)
Using mean with shape: (3L, 32L, 32L)


Running toolbox. Push h for help or q to quit.


Getting image list...
Getting image list... Done.
InputImageFetcher: bind_camera starting


WARNING: InputImageFetcher: capture device 0 failed to open! Camera will not be available!


InputImageFetcher: bind_camera finished
Starting app: CaffeVisApp
  Prettified layer name: "pool2_1" -> "p2_1"
  Prettified layer name: "pool4_2" -> "p4_2"
  Prettified layer name: "poolcp6" -> "pcp6"
CaffeProcThread.run called
I1017 09:39:36.674753 16676 common.cpp:36] System entropy source not available, using fallback algorithm to generate seed instead.
CaffeVisApp mode (in CaffeProcThread): CPUJPGVisLoadingThread.run called

Traceback (most recent call last):
  File "G:\Yosinski\deep-visualization-toolbox\run_toolbox.py", line 34, in <module>
    main()
  File "G:\Yosinski\deep-visualization-toolbox\run_toolbox.py", line 29, in main
    lv.run_loop()
  File "G:\Yosinski\deep-visualization-toolbox\live_vis.py", line 264, in run_loop
    imshow_needed |= app.draw(self.panes)
  File "G:\Yosinski\deep-visualization-toolbox\caffevis\app.py", line 165, in draw
    self._draw_status_pane(panes['caffevis_status'])
  File "G:\Yosinski\deep-visualization-toolbox\caffevis\app.py", line 307, in _draw_status_pane
    print >> status2, 'Layer size: %s' % (self.state.get_layer_output_size_string())
  File "G:\Yosinski\deep-visualization-toolbox\caffevis\caffevis_app_state.py", line 658, in get_layer_output_size_string
    layer_output_size = self.get_layer_output_size(layer_def)
  File "G:\Yosinski\deep-visualization-toolbox\caffevis\caffevis_app_state.py", line 654, in get_layer_output_size
    return SiameseHelper.get_layer_output_size(self.net, self.settings.is_siamese, layer_def, self.siamese_view_mode)
  File "G:\Yosinski\deep-visualization-toolbox\siamese_helper.py", line 511, in get_layer_output_size
    data = SiameseHelper.get_single_selected_data_blob(net, layer_def, siamese_view_mode)
  File "G:\Yosinski\deep-visualization-toolbox\siamese_helper.py", line 300, in get_single_selected_data_blob
    return SiameseHelper._get_single_selected_blob(net, layer_def, siamese_view_mode, blob_selector=lambda layer_object: layer_object.data)
  File "G:\Yosinski\deep-visualization-toolbox\siamese_helper.py", line 265, in _get_single_selected_blob
    return blob_selector(net.blobs[layer_name_to_top_name(net, layer_def['name/s'])])[0]
  File "G:\Yosinski\deep-visualization-toolbox\caffe_misc.py", line 155, in layer_name_to_top_name
    if net.top_names.has_key(layer_name) and len(net.top_names[layer_name]) >= 1:
AttributeError: _Net_IdNameWrapper instance has no attribute 'has_key'

by the issues aside I have two suggestions :

1. It would be nice to see which keys activate each section ( for example instead of pressing h for help and see 'b' is used for Backprop modes, it would be much easier to place the corrosponding keys name next to the section title ( in a paranethesis for example, like this: Modes (s) Input overlay (y) Back Prop Modes (b) Back Prop View (n)
2. It would be great to have an export option where we can export different outputs (espatially histograms and other plots ) to csv files so that we can reuse them.

by the way, how do you intrepert weight correlation? how do you say one is more correlated than the other? Thanks alot and Good Job by the way

It seems like an error from caffe code net.top_names.has_key(layer_name) fails Are you using the correct caffe version?

Also, I recommend you use the suggested .prototxt.processed_by_deepvis to prevent other possible issues related to older caffe syntax

With regards to your suggestions, I've noted them for future consideration, currently kind of busy..