rknn-toolkit2
rknn-toolkit2 copied to clipboard
rockchip-linux
Unsupported ops: Counter({'Conv3DBackpropInputV2': 6})
Hi,
I am loading and building a tensorflow model middlebury_d400.pb and encounter the following error message:
--> config model
done
--> Loading tensorflow model
W:tensorflow:From /home/paul/rknn2/lib/python3.6/site-packages/tensorflow/python/tools/strip_unused_lib.py:86: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
W:tensorflow:From /home/paul/rknn2/lib/python3.6/site-packages/tensorflow/python/tools/optimize_for_inference_lib.py:113: remove_training_nodes (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.remove_training_nodes`
2022-06-21 17:19:14,267 - E - Tensorflow op [level0/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/get_warp_disparity: Conv3DBackpropInputV2] is not supported
2022-06-21 17:19:15,132 - E - Tensorflow op [level1/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/get_warp_disparity: Conv3DBackpropInputV2] is not supported
2022-06-21 17:19:16,050 - E - Tensorflow op [level2/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/get_warp_disparity: Conv3DBackpropInputV2] is not supported
2022-06-21 17:19:17,376 - E - Tensorflow op [level3/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/get_warp_disparity: Conv3DBackpropInputV2] is not supported
2022-06-21 17:19:18,631 - E - Tensorflow op [level4/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/get_warp_disparity: Conv3DBackpropInputV2] is not supported
2022-06-21 17:19:19,802 - E - Tensorflow op [level5/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/get_warp_disparity: Conv3DBackpropInputV2] is not supported
2022-06-21 17:19:25,047 - E - Unsupported ops: Counter({'Conv3DBackpropInputV2': 6})
done
--> Building model
E build: Catch exception when building RKNN model!
E build: Traceback (most recent call last):
E build: File "rknn/api/rknn_base.py", line 2621, in rknn.api.rknn_base.RKNNBase.build
E build: File "rknn/api/graph_optimizer.py", line 3756, in rknn.api.graph_optimizer.GraphOptimizer.run
E build: File "rknn/api/graph_optimizer.py", line 2877, in rknn.api.graph_optimizer.GraphOptimizer._base_optimize
E build: File "/home/paul/rknn2/lib/python3.6/site-packages/onnx/checker.py", line 93, in check_model
E build: C.check_model(model.SerializeToString())
E build: onnx.onnx_cpp2py_export.checker.ValidationError: No Op registered for Conv3DBackpropInputV2 with domain_version of 12
E build: ==> Context: Bad node spec: input: "level0/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/get_warp_disparity/input_sizes_Concat__441:0" input: "level4/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/stack/_9__cf__9:0" input: "level0/shared/prop0/WarpImageWithHypotheses/strided_slice_3:0" output: "level0/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/get_warp_disparity:0" name: "level0/shared/prop0/WarpImageWithHypotheses/get_warp1d_coordinates_from_tile_hypotheses/get_warp_disparity" op_type: "Conv3DBackpropInputV2" attribute { name: "data_format" s: "NDHWC" type: STRING } attribute { name: "dilations" ints: 1 ints: 1 ints: 1 ints: 1 ints: 1 type: INTS } attribute { name: "padding" s: "SAME" type: STRING } attribute { name: "strides" ints: 1 ints: 4 ints: 4 ints: 1 ints: 1 type: INTS }
Build model failed!
My python script of building this model as below:
import numpy as np
import cv2
from rknn.api import RKNN
tf_model = "../models/middlebury_d400.pb"
if __name__ == '__main__':
# Create RKNN object
rknn = RKNN()
# pre-process config
print('--> config model')
rknn.config(mean_values=[[127.5,127.5,127.5,127.5,127.5,127.5]], std_values=[[127.5,127.5,127.5,127.5,127.5,127.5]],target_platform='rk3566') #rk3566: no reorder, make sure the input RGB
print('done')
# Load tensorflow model
print('--> Loading tensorflow model')
ret = rknn.load_tensorflow(tf_pb=tf_model,
inputs=["input"],
outputs=["reference_output_disparity"],
input_size_list=[[1, 226, 500, 6]])
if ret != 0:
print('Load model failed!')
exit(ret)
print('done')
# Build model
print('--> Building model')
ret = rknn.build(do_quantization=False)
#ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
if ret != 0:
print('Build model failed!')
exit(ret)
print('done')
# Export rknn model
print('--> Export RKNN model')
ret = rknn.export_rknn('./model_3566.rknn')
if ret != 0:
print('Export model_3566.rknn failed!')
exit(ret)
print('done')
#ret = rknn.load_rknn('./model_3566.rknn')
# init runtime environment
print('--> Init runtime environment')
#ret = rknn.init_runtime() #run simulator
ret = rknn.init_runtime(target='rk3566') #run inference on evb board: make sure micro usb connected and run startNPU on evb board
if ret != 0:
print('Init runtime environment failed')
exit(ret)
print('done')
print("so far so good....")
# # perf
print('--> Begin evaluate model performance')
img=torch.Tensor(226,500,6).random_(0,255)
perf_results = rknn.eval_perf(inputs=[img])
print('done')
rknn.release()
You may download the tensorflow model middlebury_d400.pb by wget command as below:
wget -P . -N https://storage.googleapis.com/tensorflow-graphics/models/hitnet/de
fault_models/middlebury_d400.pb
Could you help to provide information on how to fix this error? Thank you.
