netron icon indicating copy to clipboard operation
netron copied to clipboard

Feature visualization

Open ghylander opened this issue 4 years ago • 4 comments

add feature visualization to netron

you select/upload a image from the file system. When you sleect a layer, netron displays the features that maximise the activation of the selected layer

based on the following research paper: https://arxiv.org/pdf/1311.2901.pdf

there was a similar repo (https://github.com/keplr-io/quiver), but it was written for tensorflow 1 and it fails to construct valid maps of the models constructed with modern tools

this might be network-specific, so maybe implement slowly?

ghylander avatar Feb 24 '21 08:02 ghylander

@ghylander can you implement this and submit a pull request.

lutzroeder avatar Feb 24 '21 08:02 lutzroeder

i can try to implement the feature visualization itself for caffe and maybe keras models, but i lack the knowledge to do the front end stuff (select file, show activations on the app in the layer information tab)

ghylander avatar Feb 24 '21 13:02 ghylander

@ghylander how would this work for Caffe or Keras running in the browser and without a Python backend? Can you make sure this works for all frameworks.

lutzroeder avatar Feb 25 '21 03:02 lutzroeder

ah i see, yes, it wouldn't be possible to have it runing in the browser, since deconvolution/gradient ascent operations are required i'm developing the feature visualization for keras models using Francois Chollet's "Deep learning with python" book as reference this implmentation requires use of python and keras as the user is basically running the model in inference model reagarding all the frameworks, it would require a lot of work as i'm sure you know, each frameworks does things in their own way. Feature visualization in caffe models is not the same as in keras. They dont have the same functions, models are structured differently... it would require a per-framework function

ghylander avatar Mar 05 '21 10:03 ghylander