tf_cnnvis
tf_cnnvis copied to clipboard
infocusp
AttributeError: 'NoneType' object has no attribute 'outer_context'
Hi, I am trying to visualize inception_resent_v2 layers.
is_loaded = tf_cnnvis.activation_visualization(graph_or_path= detection_graph,
value_feed_dict={image_tensor: image_np_expanded},
layers= 'p',
path_logdir= data_dir + "results/act",
path_outdir= data_dir + "results/act")
is_loaded = tf_cnnvis.deconv_visualization(graph_or_path= detection_graph,
value_feed_dict={image_tensor: image_np_expanded},
layers= 'p',
path_logdir= data_dir + "results/rec",
path_outdir= data_dir + "results/rec")
And that is how I load the Graph;
# Load a (frozen) Tensorflow model into memory
detection_graph = tf.Graph()
with detection_graph.as_default():
od_graph_def = tf.GraphDef()
with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
serialized_graph = fid.read()
od_graph_def.ParseFromString(serialized_graph)
tf.import_graph_def(od_graph_def, name='')
I use tf_cnnvis twice, first one to visualize the activation layers, and second one for deconstruction.
The first one is working successfully, but when I try to deconstruction I get the following error;
AttributeError: 'NoneType' object has no attribute 'outer_context'
any idea what is wrong?
Regards
Hi @HeminQadir
There's an open issue in TensorFlow of this error occurring when you import graph using the above method, since it is unable to compute gradients. See: this link One solution would be to import the graph using tf.train.import_meta_graph Let us know if that works out for you.
Thanks!
Hi @falaktheoptimist
I used tf.train.import_meta_graph but this time I have this error
File "Layer_visualization.py", line 73, in <module>
path_outdir= "/home/hemin/Desktop/18594_LR_0_001/results/" + str(image))
File "build/bdist.linux-x86_64/egg/tf_cnnvis/tf_cnnvis.py", line 383, in deconv_visualization
File "build/bdist.linux-x86_64/egg/tf_cnnvis/tf_cnnvis.py", line 143, in _get_visualization
File "build/bdist.linux-x86_64/egg/tf_cnnvis/tf_cnnvis.py", line 202, in _visualization_by_layer_type
File "build/bdist.linux-x86_64/egg/tf_cnnvis/tf_cnnvis.py", line 263, in _visualization_by_layer_name
File "build/bdist.linux-x86_64/egg/tf_cnnvis/tf_cnnvis.py", line 310, in _deconvolution
File "/home/hemin/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/home/hemin/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 984, in _run
self._graph, fetches, feed_dict_string, feed_handles=feed_handles)
File "/home/hemin/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 410, in __init__
self._fetch_mapper = _FetchMapper.for_fetch(fetches)
File "/home/hemin/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 230, in for_fetch
return _ListFetchMapper(fetch)
File "/home/hemin/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 337, in __init__
self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
File "/home/hemin/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 227, in for_fetch
(fetch, type(fetch)))
TypeError: Fetch argument None has invalid type <type 'NoneType'>
Hi! You need to pass in the input_tensor argument too. You can get the tensor from the graph using get_tensor_by_name() function as explained here. This should solve your problem.
Thanks!
Hi @falaktheoptimist
I actually did that, but I am still getting the same error. I have also tried all other suggestions in here , but same error keeps happening
Here is what I have:
image_tensor = tf.get_default_graph().get_tensor_by_name('image_tensor:0')
# reading sample image
im = np.expand_dims(imresize(imread(os.path.join("/media/hemin/Data/tohongak/ETIS-LaribPolypDB/", str(image)+".tif")), (224, 224)), axis = 0)
is_working = deconv_visualization(graph_or_path= tf.get_default_graph(),
value_feed_dict={image_tensor: im},
input_tensor= tf.reshape(image_tensor, (244,244)),
layers= 'p',
path_logdir= "/home/hemin/Desktop/18594_LR_0_001/results/" + str(image),
path_outdir= "/home/hemin/Desktop/18594_LR_0_001/results/" + str(image))
some times I just use input_tensor= image_tensor.
I am about to submit my paper, but I need to include deconvolution visualization.
BTW, How we can cite this work?
Thanks!
Hi @HeminQadir Sorry for the late reply. You'll need to put (None, 244,244,1) in the reshape input. Please check if that works. We're writing a paper on this library. So, please just cite with a link to the library till then. Thanks
@HeminQadir @falaktheoptimist Hi, I tried tf.train.import_meta_graph, however, I think I also need to freeze the graph which output by tf.train.import_meta_graph? otherwise, the tf_cnnvis.py will raise an error:Please input a frozen graph (no variables). Or pass in the session object. Can you elaborate on how you did that? Thanks so much!
Here is my code:
detection_graph = tf.Graph()
with detection_graph.as_default():
od_graph_def = tf.GraphDef()
with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
serialized_graph = fid.read()
od_graph_def.ParseFromString(serialized_graph)
tf.import_graph_def(od_graph_def, name='')
im = np.expand_dims(imresize(imageio.imread(IMAGE_FILE), (244, 244)), axis = 0)
imgplot = plt.imshow(im[0,:,:,:])
plt.show()
layers = ['r']
start = time.time()
img = tf.gfile.GFile(IMAGE_FILE, 'rb').read()
img = tf.image.decode_jpeg(img, channels = 3)
img = tf.image.convert_image_dtype(img, dtype = tf.uint8)
dimension_tensor = tf.convert_to_tensor([244,244,3])
img = tf.reshape(img, dimension_tensor)
img = tf.expand_dims(img, axis = 0)
with detection_graph.as_default():
image_tensor = tf.get_default_graph().get_tensor_by_name('image_tensor:0')
is_success = deconv_visualization(sess_graph_path = tf.get_default_graph(), value_feed_dict = {image_tensor : im},
layers=layers,
input_tensor= img,
path_logdir=os.path.join("Log","Kitti"),
path_outdir=os.path.join("Output","Kitti"))
start = time.time() - start
print("Total Time = %f" % (start))
I'm having the same issue with mobilenet v2 ssd models:
I'm using the model.ckpt generated by export_inference_graph.py.
I tried to use tf.train.import_meta_graph, restore into a session, and pass session into sess_graph_path, but got this error:
ValueError: No variables to save
Then I tried to pass model.ckpt.meta file path(string) into sess_graph_path, and got
AttributeError: 'NoneType' object has no attribute 'outer_context'
activation_visualization works fine.
Am I using tf.train.import_meta_graph incorrectly? @BhagyeshVikani can you maybe make an example on how to use tf.train.import_meta_graph please?
Facing same issue after using tf.train.import_meta_graph with ssdlite_mobilenet_v2_coco model
Code
sess = tf.Session()
saver = tf.train.import_meta_graph('data/ssdlite_mobilenet_v2_coco_2018_05_09/model.ckpt.meta')
saver.restore(sess, 'data/ssdlite_mobilenet_v2_coco_2018_05_09/model.ckpt')
t_input = tf.get_default_graph().get_tensor_by_name('import/ToFloat:0')
deepdream_visualization(sess_graph_path = sess, value_feed_dict = {t_input: im},
layer='import/detection_scores', classes = [1, 2, 3, 4, 5],
path_logdir=path_logdir, path_outdir=path_outdir)
Error
File "python3.6/site-packages/tf_cnnvis-1.0.0-py3.6.egg/tf_cnnvis/tf_cnnvis.py", line 424, in deepdream_visualization
File "python3.6/site-packages/tf_cnnvis-1.0.0-py3.6.egg/tf_cnnvis/tf_cnnvis.py", line 133, in _get_visualization
File "python3.6/site-packages/tf_cnnvis-1.0.0-py3.6.egg/tf_cnnvis/tf_cnnvis.py", line 77, in _save_model
File "python3.6/site-packages/tensorflow/python/training/saver.py", line 1338, in __init__
self.build()
File "python3.6/site-packages/tensorflow/python/training/saver.py", line 1347, in build
self._build(self._filename, build_save=True, build_restore=True)
File "python3.6/site-packages/tensorflow/python/training/saver.py", line 1372, in _build
raise ValueError("No variables to save")
ValueError: No variables to save
@endeepak I've managed to get deconv working(sort of) with ssd mobilenet v2, thanks for your insight :)
Here's what I did:
import os
import sys
import time
import copy
import h5py
import numpy as np
from tf_cnnvis import *
import tensorflow as tf
from scipy.misc import imread, imresize
t_input = tf.placeholder(np.float32, name='FeatureExtractor/MobilenetV2/MobilenetV2/input') # define the input tensor
mean = 2/255.0
dev = 1.0
t_preprocessed = t_input * mean - dev
img = imresize(imread(os.path.join("./images", "image.jpg")), (300, 300))
im = np.expand_dims(img, axis = 0)
im_processed = im * mean - dev
layers = ["r", "p", "c"]
start = time.time()
#Pass ckpt path as a string would work
is_success = deconv_visualization(sess_graph_path = './ssdlite_meta/model.ckpt.meta', value_feed_dict = {t_input : im_processed},
layers=layers,
path_logdir=os.path.join("./Log","lite"),
path_outdir=os.path.join("./Output","lite"))
start = time.time() - start
print("Total Time = %f" % (start))
The key here is passing the checkpoint file as a string for sess_graph_path, if I pass the session I will get the same error ValueError: No variables to save
Also using FeatureExtractor/MobilenetV2/MobilenetV2/input as input tensor works, while image_tensor doesn't, error is TypeError: Fetch argument None has invalid type <class 'NoneType'>
And tring to deconv detection_scores layers gives error AttributeError: 'NoneType' object has no attribute 'outer_context'
So it looks like we can only visualize the feature extractor part at the moment?
Hello @wildpig22,
Here is the small snippet to import the coco model graph :
import tensorflow as tf
import os
# download ssdlite_mobilenet_v2_coco model if not
if not os.path.exists("./ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz"):
os.system("wget http://download.tensorflow.org/models/object_detection/ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz && tar -xvzf ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz")
with tf.gfile.FastGFile('ssdlite_mobilenet_v2_coco_2018_05_09/frozen_inference_graph.pb', 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
t_input = tf.placeholder(np.float32, name='FeatureExtractor/MobilenetV2/MobilenetV2/input') # define the input tensor
tf.import_graph_def(graph_def, {'FeatureExtractor/MobilenetV2/MobilenetV2/input':t_input})
# reading sample image
im = np.expand_dims(imresize(imread(os.path.join("./sample_images", "images.jpg")), (300, 300)), axis = 0)
# deepdream visualization
layer = "import/FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/BatchNorm/batchnorm/add_1" # replace this with layer of this choice
is_success = deepdream_visualization(sess_graph_path = tf.get_default_graph(), value_feed_dict = {t_input : im},
layer=layer, classes = [1, 2, 3, 4, 5],
path_logdir=os.path.join("Log","ssdlite_mobilenet_v2_coco"),
path_outdir=os.path.join("Output","ssdlite_mobilenet_v2_coco"))
@BhagyeshVikani were you able to get an interpretable image from deepdream_visualization of ssdlite_mobilenet_v2_coco ?
@endeepak Not yet. But we are working on that.
Can you please provide the name of the last output layer in the graph?
Thanks @BhagyeshVikani. There are 4 output layers in this graph detection_boxes, detection_scores, num_detections, detection_classes. I guess detection_scores is the logits layer before classification(softmax) and hence makes good candidate for deepdream_visualization
I found the output layer names by running summarize_graph command as shown below
$ bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph="ssdlite_mobilenet_v2_coco_2018_05_09/frozen_inference_graph.pb"
Found 1 possible inputs: (name=image_tensor, type=uint8(4), shape=[?,?,?,3])
No variables spotted.
Found 4 possible outputs: (name=detection_boxes, op=Identity) (name=detection_scores, op=Identity) (name=num_detections, op=Identity) (name=detection_classes, op=Identity)
..
..
To use with tensorflow/tools/benchmark:benchmark_model try these arguments:
bazel run tensorflow/tools/benchmark:benchmark_model -- --graph=ssdlite_mobilenet_v2_coco_2018_05_09/frozen_inference_graph.pb --show_flops --input_layer=image_tensor --input_layer_type=uint8 --input_layer_shape=-1,-1,-1,3 --output_layer=detection_boxes,detection_scores,num_detections,detection_classes
@BhagyeshVikani thanks!
I've got one question though, since the node FeatureExtractor/MobilenetV2/MobilenetV2/input is after the preprosessing block, shouldn't we do the preprocessing(normalize value to range [-1.0,1.0]) manually before passing the image in?
@BhagyeshVikani @falaktheoptimist There is a issue about
"AttributeError: 'NoneType' object has no attribute 'outer_context'"
when i try to import Faster-Rcnn resnet101 model, there is the code: `import os import sys import time import copy import h5py import numpy as np
from tf_cnnvis import *
import tensorflow as tf from scipy.misc import imread, imresize os.environ["CUDA_VISIBLE_DEVICES"] = "0"
image_tensor = tf.placeholder(np.float32, name='image_tensor') # define the input tensor
mean = 2/255.0 dev = 1.0 t_preprocessed = image_tensor * mean - dev
im = np.expand_dims(imresize(imread("/notebooks/workspace/data/1.jpg"), (300, 300)), axis = 0)
im_processed = im * mean - dev
layers = ["r", "p", "c"]
start = time.time()
#Pass ckpt path as a string would work is_success = deconv_visualization(sess_graph_path = '/notebooks/workspace/model/model.ckpt.meta', value_feed_dict = {image_tensor : im_processed}, layers=layers, path_logdir="/notebooks/workspace/tf_cnnvis/cwf/file/log2", path_outdir="/notebooks/workspace/tf_cnnvis/cwf/file/out2")
start = time.time() - start print("Total Time = %f" % (start))`
it is caused by loop calculate gradient？ and then how to solve it? thanks a lot
Use TensorFlow1.4 I got same Error. TensorFlow1.9 don`t appear.I think PR #3333 solve this problem. But I got new error #59
got error in tf_cnnvis.deconv_visualization, Tried everything given above. Can anybody help. tf_cnnvis.activation_visualization is working well .
Error : TypeError: Fetch argument None has invalid type <class 'NoneType'>
@aggpankaj2 Different CNNNet has different input_tensor and value_feed_dict. Which net you have used?
@jidebingfeng resnet only. Had you tried for deconv visualization for mobilenetv2. because i am getting some error while visulization of mobilenetv2
So, we think we now have this error finally figured out. This occurs when there is an operation/ block in the network which does not have a gradient defined. This would mean that while the forward pass (activation visulaization) would work fine, the backward pass would produce the above NoneType error as the backward propagation of the gradient would break when it reaches the layer/ op for which gradient is not defined. We tested and this was the case for the model in which such blocks were present.
Someone from this long and longstanding thread please express your confirmation / negation.
Hi, I still have the issue. First I am adding code here and down below I have listed the two issues. Of course, the first issue is NoneType object
#Here is how my placeholder looks in my class init(). CNN input shape is 864x480 def init(self,frozen_model_filename): tf.reset_default_graph() self.session = tf.Session() self.input_placeholder = tf.placeholder(tf.float32, shape=[ 1,864,480,3]) self.input_placeholder2 = tf.placeholder(tf.float32,shape=[None,864,480,3]) #Tied to see if # We use our "load_graph" function self.model = self.load_model_from_meta(vad_model_file_fullpath)
im = np.expand_dims(imresize(imresize(imread(os.path.join("sample_images", "mylabel1.jpg")), (256,256)) - mean, (864, 480)), axis = 0)
#visualization function layers = ['r', 'p', 'c'] start = time.time() print("before session") self.session.run(tf.global_variables_initializer())
# works and generates the folders and images.
is_success = activation_visualization(sess_graph_path = self.session, value_feed_dict = {self.input_placeholder : im},
layers=layers, path_logdir=os.path.join("Log","MyModelfromMeta"),
path_outdir=os.path.join("Output","MyModelfromMeta"))
start = time.time() - start
print("Total Time = %f" % (start))
# deconv visualization
layers = ['r', 'p', 'c']
start = time.time()
softmax_tensor = tf.get_variable('softmax_tensor')
is_success = deconv_visualization(sess_graph_path = self.session, value_feed_dict = {self.input_placeholder : im},
input_tensor=softmax_tensor,
layers=layers, path_logdir=os.path.join("Log","MyModelfromMeta"),
path_outdir=os.path.join("Output","MyModelfromMeta"))
Issue 1:- If I do not use input_tensor, then I get this error - “TypeError: Fetch argument None has invalid type <class 'NoneType'>”
Issue 2:-
If I use the input-tensor parameter, I get the following error at tf.get_variable() -
“ ValueError: Shape of a new variable (softmax_tensor) must be fully defined, but instead was
Code from model.py
#.. various layers go here. predictions = { "classes": tf.argmax(input=logits, axis=1), "probabilities": tf.nn.softmax(logits, name="softmax_tensor") }
Thanks in advance for your suggestions. Ishwar
With regard to the above issue comment, I have tried one additional api for input_tensor.
input_tensor = tf.contrib.framework.get_variables_by_name('in_layer') #tried different layers names. The names exist in the model code (which are created by tf.layer )
Error comes at tf_cnnvis.py line 282. _visualization_by_layer_name -
X = get_tensor(graph = graph, name = input_tensor.name)
AttributeError: 'list' object has no attribute 'name'
Just in case you suspect the way I built model, here are the final layer and model function.
logits = tf.layers.dense(inputs=dropout, units=logits_len,name="dense_final")
predictions = {
"classes": tf.argmax(input=logits, axis=1),
"probabilities": tf.nn.softmax(logits, name="softmax_tensor")
}
# When (trained) model is used for prediction, ie when mode == 'infer'
if mode == tf.estimator.ModeKeys.PREDICT:
export_outputs = {
'frame_pred': tf.estimator.export.PredictOutput(predictions)
}
return tf.estimator.EstimatorSpec(
mode=mode,
predictions=predictions,
export_outputs=export_outputs
)
# Calculate Loss (for 'train' or 'eval' mode).
loss = tf.losses.sparse_softmax_cross_entropy(
labels=labels,
logits=logits
)
# Logging hook
logging_hook = tf.train.LoggingTensorHook({"loss": loss}, every_n_iter=500)
# Configure the training operation (for 'train' mode)
if mode == tf.estimator.ModeKeys.TRAIN:
# Use Gradient Descent optimizer.
# optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)
optimizer = tf.train.AdamOptimizer(learning_rate=0.001)
# Minimize the loss.
train_op = optimizer.minimize(
loss=loss,
global_step=tf.train.get_global_step()
)
return tf.estimator.EstimatorSpec(
mode=mode,
loss=loss,
train_op=train_op,
training_hooks=[logging_hook]
)
# If mode == tf.estimator.Modekeys.EVAL, do the below steps.
# Add evaluation metrics for 'eval' mode
eval_metric_ops = {
"accuracy": tf.metrics.accuracy(
labels=labels, predictions=predictions["classes"])
}
return tf.estimator.EstimatorSpec(
mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)
Thanks Ishwar