facenet icon indicating copy to clipboard operation
facenet copied to clipboard

ValueError: Node 'gradients/InceptionResnetV1/Bottleneck/BatchNorm/cond/FusedBatchNorm_1_grad/FusedBatchNormGrad' has an _output_shapes attribute inconsistent with the GraphDef for output #3: Dimension 0 in both shapes must be equal, but are 0 and 512. Shapes are [0] and [512].

Open thoongee opened this issue 2 years ago • 4 comments

Hi, I'm trying to test validate_on_lfw following Wiki. I'm done align the LFW dataset, and I have a hard time on number 6. Run the test.

My computer is Ubuntu Linux. How can I run the test correctly?

2022-07-11 18:15:04.304549: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_UNKNOWN: unknown error 2022-07-11 18:15:04.304574: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: ubuntu 2022-07-11 18:15:04.304579: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: ubuntu 2022-07-11 18:15:04.304641: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 510.47.3 2022-07-11 18:15:04.304655: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 510.47.3 2022-07-11 18:15:04.304660: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 510.47.3 2022-07-11 18:15:04.304854: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. /home/facenet/src/lfw.py:83: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray. return np.array(pairs) WARNING:tensorflow:From /home/facenet/src/facenet.py:114: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version. Instructions for updating: tf.py_func is deprecated in TF V2. Instead, there are two options available in V2. - tf.py_function takes a python function which manipulates tf eager tensors instead of numpy arrays. It's easy to convert a tf eager tensor to an ndarray (just call tensor.numpy()) but having access to eager tensors means tf.py_functions can use accelerators such as GPUs as well as being differentiable using a gradient tape. - tf.numpy_function maintains the semantics of the deprecated tf.py_func (it is not differentiable, and manipulates numpy arrays). It drops the stateful argument making all functions stateful.

WARNING:tensorflow:From /home/facenet/src/facenet.py:133: batch_join (from tensorflow.python.training.input) is deprecated and will be removed in a future version. Instructions for updating: Queue-based input pipelines have been replaced by tf.data. Use tf.data.Dataset.interleave(...).batch(batch_size) (or padded_batch(...) if dynamic_pad=True). WARNING:tensorflow:From /home/heaan-venv/lib/python3.8/site-packages/tensorflow/python/training/input.py:732: QueueRunner.init (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version. Instructions for updating: To construct input pipelines, use the tf.data module. WARNING:tensorflow:From /home/heaan-venv/lib/python3.8/site-packages/tensorflow/python/training/input.py:732: add_queue_runner (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version. Instructions for updating: To construct input pipelines, use the tf.data module. Model directory: /home/models/facenet/20180402-114759 Metagraph file: model-20180402-114759.meta Checkpoint file: model-20180402-114759.ckpt-275 2022-07-11 18:15:06.172482: W tensorflow/core/common_runtime/graph_constructor.cc:1526] Importing a graph with a lower producer version 24 into an existing graph with producer version 1087. Shape inference will have run different parts of the graph with different producer versions. Traceback (most recent call last): File "/home/heaan-venv/lib/python3.8/site-packages/tensorflow/python/framework/importer.py", line 500, in _import_graph_def_internal results = c_api.TF_GraphImportGraphDefWithResults( tensorflow.python.framework.errors_impl.InvalidArgumentError: Node 'gradients/InceptionResnetV1/Bottleneck/BatchNorm/cond/FusedBatchNorm_1_grad/FusedBatchNormGrad' has an _output_shapes attribute inconsistent with the GraphDef for output #3: Dimension 0 in both shapes must be equal, but are 0 and 512. Shapes are [0] and [512].

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "src/validate_on_lfw.py", line 167, in main(parse_arguments(sys.argv[1:])) File "src/validate_on_lfw.py", line 75, in main facenet.load_model(args.model, input_map=input_map) File "/home/facenet/src/facenet.py", line 383, in load_model saver = tf.train.import_meta_graph(os.path.join(model_exp, meta_file), input_map=input_map) File "/home/heaan-venv/lib/python3.8/site-packages/tensorflow/python/training/saver.py", line 1582, in import_meta_graph return _import_meta_graph_with_return_elements(meta_graph_or_file, File "/home/heaan-venv/lib/python3.8/site-packages/tensorflow/python/training/saver.py", line 1603, in _import_meta_graph_with_return_elements meta_graph.import_scoped_meta_graph_with_return_elements( File "/home/heaan-venv/lib/python3.8/site-packages/tensorflow/python/framework/meta_graph.py", line 804, in import_scoped_meta_graph_with_return_elements imported_return_elements = importer.import_graph_def( File "/home/heaan-venv/lib/python3.8/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func return func(*args, **kwargs) File "/home/heaan-venv/lib/python3.8/site-packages/tensorflow/python/framework/importer.py", line 404, in import_graph_def return _import_graph_def_internal( File "/home/heaan-venv/lib/python3.8/site-packages/tensorflow/python/framework/importer.py", line 505, in _import_graph_def_internal raise ValueError(str(e)) ValueError: Node 'gradients/InceptionResnetV1/Bottleneck/BatchNorm/cond/FusedBatchNorm_1_grad/FusedBatchNormGrad' has an _output_shapes attribute inconsistent with the GraphDef for output #3: Dimension 0 in both shapes must be equal, but are 0 and 512. Shapes are [0] and [512].

thoongee avatar Jul 11 '22 09:07 thoongee

For those who undergo this problem. I would suggest following actions:

  1. Add the directive "import tensorflow.compat.v1 as tf" to the corresponding .py files.
  2. Use the arguments to specify the model file and pair.txt with absolute full path, as following,

python3 FaceNet/src/validate_on_lfw.py ../Inventory/Aligned /Users/xxxx/Projects/Inventory/Models/20180402-114759.pb --distance_metric 1 --use_flipped_images --subtract_mean --use_fixed_image_standardization --lfw_pairs /Users/xxxx/Projects//FaceNet/data/pairs.txt

halfroad avatar Aug 13 '22 17:08 halfroad

For those who encounter this problem, please refer to this link https://github.com/openvinotoolkit/openvino/pull/11078

michealChin avatar Apr 14 '23 23:04 michealChin

HELP! I got the same issue yet still can not fix it. I have tried different ways :((

HSSH2110 avatar Jun 22 '23 09:06 HSSH2110

same issue cant fix it :(

omkar3102 avatar Oct 20 '23 12:10 omkar3102