facenet_mtcnn_to_mobile icon indicating copy to clipboard operation
facenet_mtcnn_to_mobile copied to clipboard

代码中在导出PNET的模型会指定了输入图片的大小,如何保存PNET模型来支持可变的大小呢?

Open zjd1988 opened this issue 6 years ago • 19 comments

zjd1988 avatar Mar 20 '19 12:03 zjd1988

固化PNet时: image_op = tf.placeholder(tf.float32, shape=(1, None, None, 3), name='input') 将输入的宽高设为None就可以自适应图片了

dear-john avatar Mar 22 '19 06:03 dear-john

@dear-john 非常感谢,我按你说的,重新固化了模型,保存成功后,为了测试下保存模型的正确性,我使用tensorflow加载模型,并输入图像,但是提示下面的信息

Exception has occurred: tensorflow.python.framework.errors_impl.InvalidArgumentError You must feed a value for placeholder tensor 'input_1' with dtype float and shape [1,?,?,3] [[node input_1 (defined at F:\tensorflow_model_learn\mtcnn_learn\MTCNN-Tensorflow-master\Detection\fcn_detector.py:69) = Placeholderdtype=DT_FLOAT, shape=[1,?,?,3], _device="/job:localhost/replica:0/task:0/device:CPU:0"]] Caused by op 'input_1', defined at: File "c:\Users\ADMIN.vscode\extensions\ms-python.python-2019.2.5558\pythonFiles\ptvsd_launcher.py", line 45, in main(ptvsdArgs) File "c:\Users\ADMIN.vscode\extensions\ms-python.python-2019.2.5558\pythonFiles\lib\python\ptvsd_main_.py", line 357, in main run() File "c:\Users\ADMIN.vscode\extensions\ms-python.python-2019.2.5558\pythonFiles\lib\python\ptvsd_main_.py", line 257, in run_file runpy.run_path(target, run_name='main') File "D:\Python35\lib\runpy.py", line 254, in run_path pkg_name=pkg_name, script_name=fname) File "D:\Python35\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "D:\Python35\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "f:\tensorflow_model_learn\mtcnn_learn\MTCNN-Tensorflow-master\test\camera_test_pb_model.py", line 24, in PNet = FcnDetectorPB(P_Net, model_path_pnet) File "F:\tensorflow_model_learn\mtcnn_learn\MTCNN-Tensorflow-master\Detection\fcn_detector.py", line 69, in init _ = tf.import_graph_def(output_graph_def, name="") File "D:\Python35\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func return func(*args, **kwargs) File "D:\Python35\lib\site-packages\tensorflow\python\framework\importer.py", line 442, in import_graph_def _ProcessNewOps(graph) File "D:\Python35\lib\site-packages\tensorflow\python\framework\importer.py", line 234, in _ProcessNewOps for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access File "D:\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 3440, in _add_new_tf_operations for c_op in c_api_util.new_tf_operations(self) File "D:\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 3440, in for c_op in c_api_util.new_tf_operations(self) File "D:\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 3299, in _create_op_from_tf_operation ret = Operation(c_op, self) File "D:\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 1770, in init self._traceback = tf_stack.extract_stack() InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input_1' with dtype float and shape [1,?,?,3] [[node input_1 (defined at F:\tensorflow_model_learn\mtcnn_learn\MTCNN-Tensorflow-master\Detection\fcn_detector.py:69) = Placeholderdtype=DT_FLOAT, shape=[1,?,?,3], _device="/job:localhost/replica:0/task:0/device:CPU:0"]] File "F:\tensorflow_model_learn\mtcnn_learn\mtcnn-tensorflow-master\detection\fcn_detector.py", line 81, in predict cls_prob, bbox_pred = self.sess.run([self.cls_prob, self.bbox_pred],feed_dict={self.image_op: databatch}) File "F:\tensorflow_model_learn\mtcnn_learn\mtcnn-tensorflow-master\detection\mtcnndetector.py", line 221, in detect_pnet cls_cls_map, reg = self.pnet_detector.predict(im_resized) File "F:\tensorflow_model_learn\mtcnn_learn\mtcnn-tensorflow-master\detection\mtcnndetector.py", line 371, in detect boxes, boxes_c, _ = self.detect_pnet(img) File "F:\tensorflow_model_learn\mtcnn_learn\mtcnn-tensorflow-master\test\camera_test_pb_model.py", line 44, in boxes_c,landmarks = mtcnn_detector.detect(image)

zjd1988 avatar Mar 23 '19 07:03 zjd1988

@dear-john 你好,出现上面的错误是因为保存的模型里面有两个placeholder ,一个为input,另外一个是input_1,我只指定了其中的input。但是为什么会出现两个输入呢,在网络模型的定义的地方,并没有看到input_1?

zjd1988 avatar Mar 24 '19 02:03 zjd1988

这是我固化PNet的函数: def freeze_pnet(): with tf.Graph().as_default(): with tf.Session() as sess: image_op = tf.placeholder(tf.float32, shape=(1, None, None, 3), name='input') prob, bbox_pred, _ = PNet(image_op) output_node_names = 'cls_prob,bbox_pred,landmark_pred' saver = tf.train.Saver() saver.restore(sess, './PNet_landmark/PNet-18') output_graph_def = graph_util.convert_variables_to_constants( sess=sess, input_graph_def=sess.graph.as_graph_def(), output_node_names=output_node_names.split(",") ) with tf.gfile.GFile('pnet2.pb', 'wb') as f: f.write(output_graph_def.SerializeToString())

这是我的PNet模型: `def PNet(inputs): with slim.arg_scope([slim.conv2d], activation_fn=prelu, weights_initializer=slim.xavier_initializer(), biases_initializer=tf.zeros_initializer(), weights_regularizer=slim.l2_regularizer(0.0005), padding='valid'): net = slim.conv2d(inputs, num_outputs=10, kernel_size=3, stride=1, scope='conv1') net = slim.max_pool2d(net, 2, 2, padding='SAME', scope='pool1') net = slim.conv2d(net, 16, 3, 1, scope='conv2') net = slim.conv2d(net, 32, 3, 1, scope='conv3')

    conv4_1 = slim.conv2d(net, 2, 1, 1, scope='conv4_1', activation_fn=tf.nn.softmax)
    bbox_pred = slim.conv2d(net, 4, 1, 1, scope='conv4_2', activation_fn=None)
    landmark_pred = slim.conv2d(net, 10, 1, 1, scope='conv4_3', activation_fn=None)

    # use pre-train model because train from scratch is so hard
    # # when test, batch size = 1, so squeeze dim
    cls_prob = tf.squeeze(conv4_1, axis=0, name='cls_prob')
    bbox_pred = tf.squeeze(bbox_pred, axis=0, name='bbox_pred')
    landmark_pred = tf.squeeze(landmark_pred, axis=0, name="landmark_pred")
    return cls_prob, bbox_pred, landmark_pred`

我这里定义的placeholder是仿照别人写的,我感觉加这个placeholder就是为了覆盖真正的输入,因为我使用的ckpt文件的输入是2561212*3的,没法自定义大小,你可以看看你自己的ckpt文件有没有这个问题。

dear-john avatar Mar 24 '19 03:03 dear-john

@dear-john 非常感谢,问题解决了。你提供的模型保存代码没问题,是我的PB模型验证代码的问题, 代码里面我使用placeholder新建input,而不是直接使用get_tensor_by_name获取的input,导致出问题。

目前模型验证通过了,谢谢啦!

zjd1988 avatar Mar 25 '19 00:03 zjd1988

@anneyking 目前tensorflow lite的模型是输入的图片的尺寸是固定的。

zjd1988 avatar May 28 '19 06:05 zjd1988

@anneyking 我记不太清了,你可以使用一些网络模型查看的工具进行查看。如何发你呢?

zjd1988 avatar May 28 '19 06:05 zjd1988

@zjd1988 hi, are you going to share your code importing tflite model and run it with python? my porting to android fails.

flankechen avatar May 31 '19 13:05 flankechen

@zjd1988 @dear-john 请问你们固化1NoneNone*3作为Pnet的input之后,如何把这个.pb转化到tflite文件呢,

to_tflite.sh里面的命令 tflite_convert --output_file MTCNN_mobile/pnet2.tflite --graph_def_file frozen_graphs/pnet2.pb --input_arrays "input" --input_shapes "1,600,800,3" --output_arrays cls_prob,bbox_pred,landmark_pred --output_format TFLITE 这个input_shapes怎么写呢, 如果我不写, 会得到这个错误 ValueError: None is only supported in the 1st dimension. Tensor 'input' has invalid shape '[1, None, None, 3]'.

如果需要固定input_shape,如何实现原来的fcn的网络和结果呢

flankechen avatar Jun 04 '19 09:06 flankechen

Pnet网络模型的输入大小固定后,我采用对输入图片缩放到指定大小,然后再做推断。网上还有一种办法是将缩放后的几个图片拼接为一个图片,然后作为输入进行推断。

zjd1988 avatar Jun 17 '19 06:06 zjd1988

@zjd1988 那好像,金字塔就只能用缩放后的几个拼成一个这种做法实现了也。。。好像有点麻烦,唉

flankechen avatar Jun 18 '19 06:06 flankechen

Pnet网络模型的输入大小固定后,我采用对输入图片缩放到指定大小,然后再做推断。网上还有一种办法是将缩放后的几个图片拼接为一个图片,然后作为输入进行推断。

你好,请问你的意思是不是说到最后tflite模型的时候图片大小必须得固定,除了拼接金字塔的方法外没有办法像常规那样生成若干张不同大小的图片输入P-Net是吗?

TMaysGGS avatar Aug 06 '19 08:08 TMaysGGS

@TMaysGGS 是的,还有一种方法就是你生成不同输入的几种Pnet模型,不同大小图像使用不同的Pnet模型

zjd1988 avatar Aug 08 '19 07:08 zjd1988

@TMaysGGS 是的,还有一种方法就是你生成不同输入的几种Pnet模型,不同大小图像使用不同的Pnet模型

好的非常感谢,这么看的话可能拼接图像更合适,不过之前似乎用拼接图像方法的话pooling必须先精准计算过,多一圈都会导致结果不对……

TMaysGGS avatar Aug 09 '19 07:08 TMaysGGS

感觉使用tensorflow lite的性能不是很好,当时在树梅派上有简单测试过

zjd1988 avatar Aug 09 '19 08:08 zjd1988

感觉使用tensorflow lite的性能不是很好,当时在树梅派上有简单测试过

这样啊,好的谢谢,我也发现tflite的模型前向输出的结果跟我原来的Keras模型结果有些相差,毕竟是回归问题本来是希望越精确越好的……性能如果说是速度的话是不是要看最终部署到什么上去决定吧?

TMaysGGS avatar Aug 12 '19 05:08 TMaysGGS

@TMaysGGS 是的,我当时是在raspberry pi测试的,我以为你要在终端部署你的模型了,毕竟这个respository介绍的是facenet_mtcnn_to_mobile.

zjd1988 avatar Aug 12 '19 06:08 zjd1988

@TMaysGGS 是的,我当时是在raspberry pi测试的,我以为你要在终端部署你的模型了,毕竟这个respository介绍的是facenet_mtcnn_to_mobile.

好的好的,感谢。我是遇到了转tflite模型不得不固定输入大小的问题时搜到这个issue的。

TMaysGGS avatar Aug 12 '19 06:08 TMaysGGS

@TMaysGGS 想要形状不固定,freeze_graph这一步输入的形状需要设置为None,然后在tflite_convert时虽然必须输入形状,但是转成.tflite之后input是可以不固定的。

感谢回复,我这么做似乎在转换为tflite的时候会报错……所以后来还是采用固定输入的解决办法了。

TMaysGGS avatar Nov 12 '19 07:11 TMaysGGS