Train and Test R-CNN on Another Dataset

[zeyuanxy]

Hi Ross, Fast-RCNN is really fantastic! I am impressed by its great performance and speed, thus I add some other code and two READMEs to help train and test Fast-RCNN on INRIA Person, and it is easy to modify it to train on other datasets, not limiting to PASCAL VOC. It is really a honor for me to help with this, Thanks!

[rbgirshick]

Hi @zeyuanxy, I'm curious to know what mAP you get on INRIA?

I won't be able to merge this PR in its current form because it includes the selective search code, which is under a license that does not permit redistribution.

[zeyuanxy]

Hi @rbgirshick It is about 87.8, a very high number.

[cuatristapr]

@zeyuanxy Do you have a trained model that I can already use in my project?

[futurely]

You don't have to stick with the MATLAB implementation of selective search. There's a pure python version.

Usage is a super easy one-liner rects = dlibss.selective_search(img, 50, 200, 3, 20, 50).

[cuatristapr]

How do I use it futurely? Where do I add that line to point to the python wrapper of Selective Search?

[sunshineatnoon]

@zeyuanxy Hi~, How did you generate images for the background class? How many images are needed for this class?

[zeyuanxy]

@sunshineatnoon I am sorry that I did not remember that. Maybe I have not used any background images.

On Mon, Sep 14, 2015 at 3:19 AM, SunshineAtNoon [email protected] wrote:

@zeyuanxy https://github.com/zeyuanxy Hi~, How did you generate images for the background class? How many images are needed for this class?

— Reply to this email directly or view it on GitHub https://github.com/rbgirshick/fast-rcnn/pull/21#issuecomment-139982672.

Best Regards, Zeyuan Shang Department of Computer Science & Technology Tsinghua University Mobile: +86 15635986068

[zeyuanxy]

@cuatristapr I am sorry that the models were stored on the server of the company I interned. You can follow the steps I have provided and it is easy to reproduce one.

[DaChaoXc]

@zeyuanxy Hi,I haved tried your mothds,but i have some problems.Did you meet this? File "./tools/train_net.py", line 80, in roidb = get_training_roidb(imdb) File "/home/xc/fast-rcnn-master/tools/../lib/fast_rcnn/train.py", line 107, in get_training_roidb imdb.append_flipped_images() File "/home/xc/fast-rcnn-master/tools/../lib/datasets/imdb.py", line 99, in append_flipped_images boxes = self.roidb[i]['boxes'].copy() File "/home/xc/fast-rcnn-master/tools/../lib/datasets/imdb.py", line 63, in roidb self._roidb = self.roidb_handler() File "/home/xc/fast-rcnn-master/tools/../lib/datasets/inria.py", line 116, in selective_search_roidb ss_roidb = self._load_selective_search_roidb(gt_roidb) File "/home/xc/fast-rcnn-master/tools/../lib/datasets/inria.py", line 138, in _load_selective_search_roidb return self.create_roidb_from_box_list(box_list, gt_roidb) File "/home/xc/fast-rcnn-master/tools/../lib/datasets/imdb.py", line 167, in create_roidb_from_box_list argmaxes = gt_overlaps.argmax(axis=1) ValueError: attempt to get argmax of an empty sequence

[xiaoqiqi177]

@DaChaoXc I just met the same problem when I do training(btw, I tried to train INRIA). Have you solved this problem? Thank you in advance!

[DaChaoXc]

@nicci1771 NO,i am still confused.

[pradeepj247]

@zeyuanxy, @rbgirshick

I have a naive question.on training on custom dataset

i have got the installation done and got the demo.py to work successfully

i have read about the alternating optimization also and my query is about training fast rCNN on a custom dataset.

let's say I have oxford flowers dataset, for which I would be training a model using AlexNet. And I want to use the power of region proposals to improve accuracy.

Specifially, my question is on the region proposals - since they would come from the training done on PASCAL VOC 2007 dataset. But my detections are going to be on Flowers.

Does it even matter? How does learning regions on a VOC dataset, help in proposing for Flowers or some other objects on a different model.

What is the correct way of understanding this and resolving. Can someone pls help here?

many thanks

[futurely]

Custom datasets need to implement their own imdb classes following the PASCAL VOC examples. https://github.com/rbgirshick/fast-rcnn/blob/master/lib/datasets/imdb.py https://github.com/rbgirshick/fast-rcnn/blob/master/lib/datasets/pascal_voc.py https://github.com/rbgirshick/fast-rcnn/blob/master/lib/datasets/factory.py

[pradeepj247]

thanks @futurely, @zeyuanxy,

I found the relevant documentation for creating the 3 python files as suggested and I have also created the dataset as suggested in the INRIA example.

Now, in what sequence should I run these 3 Python files? and after running them I should call the train net ?

[futurely]

https://github.com/rbgirshick/fast-rcnn/tree/master/experiments

[pradeepj247]

@futurely,

thanks for responding regularly. this is exciting and it feels like am on a treasure hunt and you dropping hints to help move further along!

but honestly this time, i really am unable to connect the dots from the 3 python files to these scripts.

am going to use the all_caffenet.sh but i don't quite see where the earlier mentioned python files (imdb, factory etc.) are being invoked from the scripts inside this one. and also I didn't get how to use the cfgs

[sunshineatnoon]

@pradeepj247 these links might help:

https://github.com/zeyuanxy/fast-rcnn/tree/master/help/train http://sunshineatnoon.github.io/Train-fast-rcnn-model-on-imagenet-without-matlab/

[pradeepj247]

@sunshineatnoon thanks!

this blog is informative. i have one question on the dataset preparation.

in train.txt, i see that you have the entries in the form of "classname_image#". So if we have 20,000 images belonging to 200 classes, then each image should be listed here in this way along with the class it belongs to?

and about the full path to the images - i presume we mention it through Factory.py?

thanks

[sunshineatnoon]

@pradeepj247 You don't need to mention which class the image belongs to in the train.txt file, its only the names of the images, if you see classes included in images' names, that's pure coincidence. The annotation files should indicate which class a image belongs to. For instance, a pascal voc annotation file looks like this:

<object>
        <name>dog</name>
        <pose>Left</pose>
        <truncated>1</truncated>
        <difficult>0</difficult>
        <bndbox>
            <xmin>48</xmin>
            <ymin>240</ymin>
            <xmax>195</xmax>
            <ymax>371</ymax>
        </bndbox>
    </object>

Then the name tag tells you which class this object belongs to.

Yes, you need to mention the full path to the images in factory.py. All these three files we modified tell fast rcnn how to find data and how to parse the annotation files.

[pradeepj247]

@sunshineatnoon, just 2 more questions:

I) in your Factory.py, there are 3 for..loops outside the functions, and they contain, the imagenet_devkit_path = '/home/xuetingli/imagenet' and towncenter_devkit_path = '/home/szy/TownCenter' and also inria_devkit_path = '/home/xuetingli/test/INRIA'. This is a bit confusing for me. if you are dealing with only one dataset (i.e imagenet), then only the imagenet should suffice?

2. after setting up everything as mentioned by you and when I run train, i see this error: EnvironmentError: MATLAB command 'matlab' not found. Please add 'matlab' to your PATH. do we still need MATLAB? I thought this was a pure Python implementation, especially, if we bring in our own selective search proposals and train.mat

[sunshineatnoon]

1. yes, only imagenet loop is needed, you can deleted others. As I mentioned, I referred here for how to modify the code, so I didn't delete what the author wrote initially.
2. this error is caused by code in lib/datasets/init.py, so just comment it out:

if _which(MATLAB) is None:
    msg = ("MATLAB command '{}' not found. "
           "Please add '{}' to your PATH.").format(MATLAB, MATLAB)
    raise EnvironmentError(msg)'''

[pradeepj247]

Thanks @sunshineatnoon,

I crossed all those stages - it loaded the dataset, annotations and the caffeNet model required for training.

But just before starting the training, it failed with the following error: Check failed: error == cudaSuccess (8 vs. 0) invalid device

This seems to be quite a popular error and I read quite a few posts on the forums, but couldn't resolve it.

I am running this on AWS g2.2x and my caffe device_query call works fine.

Any suggestions on how to get past this one?

[KrasusC]

@DaChaoXc I met the problem you mentioned. Have you figured it out?

[leejiajun]

@DaChaoXc @KrasusC @nicci1771 I met the problem you mentioned. Have you figured it out? "./tools/train_net.py", line 80, in roidb = get_training_roidb(imdb) File "/home/xc/fast-rcnn-master/tools/../lib/fast_rcnn/train.py", line 107, in get_training_roidb imdb.append_flipped_images() File "/home/xc/fast-rcnn-master/tools/../lib/datasets/imdb.py", line 99, in append_flipped_images boxes = self.roidb[i]['boxes'].copy() File "/home/xc/fast-rcnn-master/tools/../lib/datasets/imdb.py", line 63, in roidb self._roidb = self.roidb_handler() File "/home/xc/fast-rcnn-master/tools/../lib/datasets/inria.py", line 116, in selective_search_roidb ss_roidb = self._load_selective_search_roidb(gt_roidb) File "/home/xc/fast-rcnn-master/tools/../lib/datasets/inria.py", line 138, in _load_selective_search_roidb return self.create_roidb_from_box_list(box_list, gt_roidb) File "/home/xc/fast-rcnn-master/tools/../lib/datasets/imdb.py", line 167, in create_roidb_from_box_list argmaxes = gt_overlaps.argmax(axis=1) ValueError: attempt to get argmax of an empty sequence

[nwestlake]

@leejiajun Try applying the changes proposed here: https://github.com/rbgirshick/fast-rcnn/pull/102

e.g. git fetch [email protected]:nw362/fast-rcnn.git negativeImages:negativeImages git merge negativeImages

(I am interesting in the pull request too, so I might later try merging it with mine.)

[leejiajun]

@nw362 nice work! I will fetch you code, thank you.

[aragon111]

Hello, when I try to train, I get this error

File "/home/attilio/fast-rcnn/tools/../lib/datasets/amph.py", line 190, in get_data_from_tag return node.getElementsByTagName(tag)[0].childNodes[0].data IndexError: list index out of range

Would you help to understand how to fix it? I don't know if it depends on the Annotation file I have or some editing error of python files (I use just one class)

[nwestlake]

This looks to be code you have written (amph.py)? Hard to say without seeing both the code and the annotation file.

[aragon111]

I fixed it adding to the .xml files (which I got from Matlab) the fields name and folder.

[MinaRe]

Dear All, (@zeyuanxy ) can you please kindly tell me how can I change batch_size?

[IdiosyncraticDragon]

Editing the config file FAST-RCNN-ROOT/lib/fast_rcnn/config.py

发自我的 iPhone

在 2016年4月19日，下午5:35，MinaRe [email protected] 写道：

Dear All, can you please kindly tell me how can I change batch_size?

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub

[MinaRe]

thanks @IdiosyncraticDragon Actually my training stop during "Computing bounding-box regression targets..." ,do you have any Idea? is this problem because of memory size? I have 50k images, How much CPU memory do I need for training?

[IdiosyncraticDragon]

Well, that's because of too much training data. I don't know whether it will be helpful if you have more memory size.

发自我的 iPhone

在 2016年4月19日，下午8:11，MinaRe [email protected] 写道：

thanks @IdiosyncraticDragon Actually my training stop during "Computing bounding-box regression targets..." ,do you have any Idea? is this problem because of memory size?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[aragon111]

Can you tell me why running detection, the frame detect just one item at a time?

[nwestlake]

One ROI at a time?

[aragon111]

@nw362 exactly :)

[nwestlake]

The convolutional layers of the CNN are calculated on the whole image (resized before hand). The final polling layer pools over only the ROI and so the CNN must run the forward steps over these layers for every ROI.

[aragon111]

In order to detect more ROIs in the same frame there is anything I could do?

[nwestlake]

It should happen already "lib/fast_rcnn/test.py": _get_blobs will take in one image but multiple ROIs. In im_detect(net, im, boxes), the CNN will be called forward only once for an image and a set of ROIs. Is only one ROI being sent to this method?

[aragon111]

@nw362 I checked and it seems to me that it does not send only one ROI.

[aragon111]

I have a little doubt. During the labeling of the training set, is it possible to label more items (ROIs) in the same image?

[nwestlake]

Yes, e.g. the xml format used for annotations.

[aragon111]

@nw362 Do you mean that is not a problem if there are more bounding boxes for the same .xml file?

[nwestlake]

No problem at all. This is the case for many data sets.

[aragon111]

@nw362 I'm not sure about what you mean with many data sets. I have only two classes (background and amphora) and in some image from the data set there are many amphoras.

[mksarker]

hi all, I have followed the below web instruction for training my own dataset as INRIA, https://github.com/zeyuanxy/fast-rcnn/blob/master/help/train/README.md and found the below problem, :~$ cd fast-rcnn :~/fast-rcnn$ ./tools/train_net.py --gpu 0 --solver models/pascal_voc/VGG_CNN_M_1024/fast-rcnn/solver.prototxt --weights data/faster_rcnn_models/VGG16_faster_rcnn_final.caffemodel --imdb inria_train /home/sarker/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment. warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.') Python 2.7.11 |Anaconda custom (64-bit)| (default, Jun 15 2016, 15:21:30) Type "copyright", "credits" or "license" for more information.

IPython 4.2.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.

In [1]:

when I run the training code on terminal the IPython console is open? I don’t know what is the problem? I am a beginner in this area. Please help me. Thanks in advanced....