Faster-RCNN_TF icon indicating copy to clipboard operation
Faster-RCNN_TF copied to clipboard

How to train my own model

Open yang1997yi opened this issue 7 years ago • 46 comments

I have changed the Annotation, Imagesets and JPEG dir into my dataset in VOC2007 and changed the classes and num_classes to 15. But it still don't work. Where else should I change?

Thanks.

yang1997yi avatar Jul 24 '17 07:07 yang1997yi

hi,can you tell me how to make files of the annotation dir? any tools can be used?

zjz5250 avatar Aug 04 '17 07:08 zjz5250

Hi @yang1997yi

You have to make following changes also: lib/pascal_voc.py --> change the classes tools/demo.py --> change the classes

Thanks

Akansha06 avatar Aug 09 '17 09:08 Akansha06

Hi @zjz5250

You can use LabelImage tool for generating xml files for Annotation directory.

Thanks

Akansha06 avatar Aug 09 '17 09:08 Akansha06

how to change Main and Segementation floder under ImageSets @yang1997yi

gentlebreeze1 avatar Aug 24 '17 07:08 gentlebreeze1

how to change Main and Segementation floder under ImageSets @Akansha06

gentlebreeze1 avatar Aug 24 '17 07:08 gentlebreeze1

Hi @gentlebreeze1

Suppose u have 2 classes- male and female. You should have following named files in your Main folder i.e trainval.txt -->contains name of all the images train.txt and val.txt --> split the names into 70-30 ratio and add 70% to train.txt and 30% to val.txt test.txt --> pick some random image names and put it in test.txt

For each class: male.txt --> contain names of all the male images maletrain.txt, maleval.txt --> same as mentioned above(but ds data will contain only male class images name) Then same for female class.

Note: Please don't include extension inside these txt. Only names should be added. Else you ll get an error.

Happy Learning :)

Akansha06 avatar Aug 25 '17 04:08 Akansha06

Can you send me the modified code in lib/pascal_voc.py and tools/demo.py? should i modify other floders ?my email is [email protected] OR [email protected] ,thanks! @Akansha06

gentlebreeze1 avatar Aug 26 '17 01:08 gentlebreeze1

but how to get 1 or -1 in male_train.txt?do you have code to make dataset? @Akansha06 thanks

gentlebreeze1 avatar Aug 26 '17 01:08 gentlebreeze1

AssertionError: Path does not exist: /home/c303/student/yongzhang/Faster-RCNN_TF/data/VOCdevkit2007/VOC2007/JPEGImages/.jpg what's wrong? @Akansha06

gentlebreeze1 avatar Aug 26 '17 02:08 gentlebreeze1

Hi @gentlebreeze1 ,

In any of your text files there is a blank line. For this reason only there is assertion error

artificial-mind avatar Aug 28 '17 05:08 artificial-mind

Hi @gentlebreeze1,

I am not clear with 1 and -1 . Can you explain it?? Because I have given only names in txt's and it was working fine

artificial-mind avatar Aug 28 '17 05:08 artificial-mind

The data of standard VOC2007/ImageSets/main**_train.txt is as follows: 0000005 -1 0000007 1 0000009 -1 they say that the mean of 1 is positive sample and -1 is negative sample @Akansha62

gentlebreeze1 avatar Aug 28 '17 13:08 gentlebreeze1

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [8] rhs shape= [84] [[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@bbox_pred/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](bbox_pred/biases, save/RestoreV2/_1)]] i get the problem , is it a tensorflow problem? @Akansha62

gentlebreeze1 avatar Aug 29 '17 01:08 gentlebreeze1

Hi @gentlebreeze1,

How many classes you are using?? You have to change it in VGGtest.py and VGGtrain.py present inside tools.

artificial-mind avatar Aug 29 '17 05:08 artificial-mind

Hi @gentlebreeze1 ,

You don't have to give it 1 and -1 it will simply work

artificial-mind avatar Aug 29 '17 05:08 artificial-mind

Thanks very much for your answer @Akansha62 i have 2 classes,i change classes into 3 in VGGtest.py and VGGtrain.py inside lib/networks. is the problem related to the follows? WARNING:tensorflow:******************************************************* WARNING:tensorflow:TensorFlow's V1 checkpoint format has been deprecated. WARNING:tensorflow:Consider switching to the more efficient V2 format: WARNING:tensorflow: tf.train.Saver(write_version=tf.train.SaverDef.V2) WARNING:tensorflow:now on by default. WARNING:tensorflow:******************************************************* Traceback (most recent call last): File "./tools/test_net.py", line 99, in saver.restore(sess, args.model) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1439, in restore {self.saver_def.filename_tensor_name: save_path}) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 767, in run run_metadata_ptr) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 965, in _run feed_dict_string, options, run_metadata) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1015, in _do_run target_list, options, run_metadata) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1035, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [3] rhs shape= [21] [[Node: save/Assign_2 = Assign[T=DT_FLOAT, _class=["loc:@cls_score/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](cls_score/biases, save/RestoreV2_2/_5)]]

Caused by op u'save/Assign_2', defined at: File "./tools/test_net.py", line 97, in saver = tf.train.Saver() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1051, in init self.build() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1081, in build restore_sequentially=self._restore_sequentially) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 675, in build restore_sequentially, reshape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 414, in _AddRestoreOps assign_ops.append(saveable.restore(tensors, shapes)) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 155, in restore self.op.get_shape().is_fully_defined()) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_state_ops.py", line 47, in assign use_locking=use_locking, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2395, in create_op original_op=self._default_original_op, op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1264, in init self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [3] rhs shape= [21] [[Node: save/Assign_2 = Assign[T=DT_FLOAT, _class=["loc:@cls_score/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](cls_score/biases, save/RestoreV2_2/_5)]]

gentlebreeze1 avatar Aug 29 '17 13:08 gentlebreeze1

it seems classes problem .i change pascal_voc.py,VGGnet_train.py and VGGnet_test.py.what else should i change? @Akansha62

gentlebreeze1 avatar Aug 31 '17 01:08 gentlebreeze1

Hi @gentlebreeze1 ,

Please grep any of the 21 classes . And wherever u find them change accordingly

artificial-mind avatar Aug 31 '17 05:08 artificial-mind

Hi @Akansha62 , i try again,i get different problem.as follow: File "./tools/train_net.py", line 96, in max_iters=args.max_iters) File "/home/c303/student/yongzhang/Faster-RCNN_TF/tools/../lib/fast_rcnn/train.py", line 264, in train_net sw.train_model(sess, max_iters) File "/home/c303/student/yongzhang/Faster-RCNN_TF/tools/../lib/fast_rcnn/train.py", line 198, in train_model self.snapshot(sess, iter) File "/home/c303/student/yongzhang/Faster-RCNN_TF/tools/../lib/fast_rcnn/train.py", line 61, in snapshot sess.run(net.bbox_weights_assign, feed_dict={net.bbox_weights: orig_0 * np.tile(self.bbox_stds, (weights_shape[0], 1))})

ValueError: operands could not be broadcast together with shapes (4096,16) (4096,12)

gentlebreeze1 avatar Aug 31 '17 07:08 gentlebreeze1

Hi @gentlebreeze1 ,

I guess still somewhere your number of classes are given wrong, Please check again.

grep -irn <n_classes> * grep -irn *

use dse commands and get the data, and then see somewhere you r missing something

artificial-mind avatar Aug 31 '17 07:08 artificial-mind

@Akansha62 , maybe i use the trained model that cause the problem .but new problem occurs: VOC07 metric? Yes Reading annotation for 1/170 Traceback (most recent call last): File "./tools/test_net.py", line 97, in test_net(sess, network, imdb, weights_filename) File "/home/c303/student/yongzhang/ai/Faster-RCNN_TF/tools/../lib/fast_rcnn/test.py", line 345, in test_net imdb.evaluate_detections(all_boxes, output_dir) File "/home/c303/student/yongzhang/ai/Faster-RCNN_TF/tools/../lib/datasets/pascal_voc.py", line 321, in evaluate_detections self._do_python_eval(output_dir) File "/home/c303/student/yongzhang/ai/Faster-RCNN_TF/tools/../lib/datasets/pascal_voc.py", line 284, in _do_python_eval use_07_metric=use_07_metric) File "/home/c303/student/yongzhang/ai/Faster-RCNN_TF/tools/../lib/datasets/voc_eval.py", line 109, in voc_eval recs[imagename] = parse_rec(annopath.format(imagename)) File "/home/c303/student/yongzhang/ai/Faster-RCNN_TF/tools/../lib/datasets/voc_eval.py", line 19, in parse_rec obj_struct['pose'] = obj.find('pose').text AttributeError: 'NoneType' object has no attribute 'text'

gentlebreeze1 avatar Aug 31 '17 07:08 gentlebreeze1

Thank you for your guidance during this time @Akansha62 @Akansha06 .I have been running successfully on my dataset.

gentlebreeze1 avatar Sep 01 '17 03:09 gentlebreeze1

Hi @gentlebreeze1,

Congratulations....

Happy Learning :)

artificial-mind avatar Sep 01 '17 05:09 artificial-mind

What is the meaning of this document in VOC2007\ ImageSets\Layout? should i make the txt files like train.txt,val.txt and trainval.txt? @Akansha62

gentlebreeze1 avatar Sep 01 '17 07:09 gentlebreeze1

Hi @gentlebreeze1 ,

Layout is not necessary. It just the replica of your Main folder.

artificial-mind avatar Sep 01 '17 07:09 artificial-mind

i only need to change Main floder.Train.txt in Main floder includes all classes's ***_trian.txt. @Akansha62

gentlebreeze1 avatar Sep 01 '17 07:09 gentlebreeze1

yes @gentlebreeze1

artificial-mind avatar Sep 01 '17 10:09 artificial-mind

if i want to test my pictures use demo.py.where should i change @Akansha62

gentlebreeze1 avatar Sep 01 '17 13:09 gentlebreeze1

Hi @gentlebreeze1 ,

  l have the error as follows:

Traceback (most recent call last): File "/usr/lib/python2.7/pdb.py", line 1314, in main pdb._runscript(mainpyfile) File "/usr/lib/python2.7/pdb.py", line 1233, in _runscript self.run(statement) File "/usr/lib/python2.7/bdb.py", line 400, in run exec cmd in globals, locals File "", line 1, in File "./tools/train_net.py", line 10, in """Train a Fast R-CNN network on a region of interest database.""" File "./tools/../lib/fast_rcnn/train.py", line 267, in train_net sw.train_model(sess, max_iters) File "./tools/../lib/fast_rcnn/train.py", line 198, in train_model self.snapshot(sess, iter) File "./tools/../lib/fast_rcnn/train.py", line 63, in snapshot sess.run(net.bbox_weights_assign, feed_dict={net.bbox_weights: orig_0 * np.tile(self.bbox_stds, (weights_shape[0], 1))}) ValueError: operands could not be broadcast together with shapes (4096,84) (4096,8) Uncaught exception. Entering post mortem debugging Running 'cont' or 'step' will restart the program

/home/tf/Downloads/Faster-RCNN_TF/lib/fast_rcnn/train.py(63)snapshot() -> sess.run(net.bbox_weights_assign, feed_dict={net.bbox_weights: orig_0 * np.tile(self.bbox_stds, (weights_shape[0], 1))})

 when i use the command:
 grep -irn <n_classes> *
 grep -irn *

new problem occurred: "bash: n_classes: No such file or directory"

zjz5250 avatar Sep 11 '17 06:09 zjz5250

l have solved the problem, by changing the VGGnet_train.py file as follows: n_classes = 2

zjz5250 avatar Sep 11 '17 07:09 zjz5250