TrainYourOwnYOLO
TrainYourOwnYOLO copied to clipboard
AntonMu
Training images with zero annotations
What would be the best way to modify the code to support images with zero annotations? When you export the annotations as csv from Vott it makes a new folder with All the images you checked but it leaves images without annotations absent from the csv. Which means the training script will simply ignore them.
I was thinking the "Convert_to_YOLO_format.py" would be the best place to add in the images without annotations by making it check the folder for all the images that exist and than checking to see if they are referenced in the csv and if not add a row without bounding boxes.
Do you think this would be the best way? or will the training script fail if an images with no bounding boxes is provided in the dataset?
Hi Niklas - I think we would need to make sure that they are explicitly labeled as not containing the object. Because what will happen more often than not is that there are 1000s of images in the image folder and after labeling a few hundred, someone might stop labeling and then the way you describe it, the program would automatically assume that all remaining images don't have the object.
What about making an extra folder somewhere called "No_Object" in Training_Images and then simply add all files in there to the data_train.txt the way you described it.
Adding a "No_object" folder would be good however by default VoTT puts the annotated csv as well the images that have been looked at into the csv_export folder. (By default it excludes images that have yet to be checked for tags)
It could be a lot of extra labor to manually move images to a "no_object" folder as I wont know they have no object till after I look at them in VoTT. I am thinking a flag could be added to the script that defaults to false like --check_for_no_annotations or something similiar/shorter.
Adding a no object folder as well that is checked by default would be cool though.
Ok the folder path actually will work for both of us. I can just pass in the folder where the annotations are. PR coming up with the change, by default it will now check for a no_object folder.
Ok cool - thanks for this!
On Mon, Nov 25, 2019 at 11:20 Niklas Wilson [email protected] wrote:
Ok the folder path actually will work for both of us. I can just pass in the folder where the annotations are. PR coming up with the change, by default it will now check for a no_object folder.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/AntonMu/TrainYourOwnYOLO/issues/43?email_source=notifications&email_token=AEEZVCXIVCX663N6PTQKJN3QVQQOZA5CNFSM4JRMCLDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFDQCKY#issuecomment-558301483, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEEZVCVTV6FPTJNGMFKJ6BDQVQQOZANCNFSM4JRMCLDA .
No problem. Check that out, obviously make sure it doesn't cause you any issues :)
It looks like it includes the last PR? Im not sure how to update a fork to resolve this, ive actually never done a PR across a fork before.
@NiklasWilson How to do the image training with zero annotation, because I just want the machine to classify whether the object present in the image or not, bounding box is unnecessary.
Hi @forceassaulter - there is some confusion here. @NiklasWilson is talking about improving the object detector by adding additional images without annotations. What you are looking for is image classification. That's a different algorithm. Check out Inception v3 for instance.
@forceassaulter : @AntonMu Is correct there is another algorithm used for classification instead of detection. However if you willing to draw annotations around stuff, there is no reason why you cant use this code to do both, in the detection script instead of drawing boxes on images just use was it detected to classify the image. That said I believe there is a huge accuracy/efficiency loss if you were to use this algorithm for pure image classification.
My additional code to accept images with no drawn boxes/annotations is not the solution you want though. It works along side images with annotations.
@NiklasWilson Other than image classification, I also need an algorithm to draw bounding boxes on object detected, so I was thinking to determine whether there is box drawn on image to determine whether the object is in the image.
Ok if your goal is to detect objects in images then yolo is the right algorithm for you. This algorithm (described here) figures out if an object is in an image and then draws a bounding box around it.
You can simply train on multiple classes and everything should work fine.
On Thu, Dec 5, 2019 at 17:38 forceassaulter [email protected] wrote:
@NiklasWilson https://github.com/NiklasWilson Other than image classification, I also need an algorithm to draw bounding boxes on object detected, so I was thinking to determine whether there is box drawn on image to determine whether the object is in the image.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/AntonMu/TrainYourOwnYOLO/issues/43?email_source=notifications&email_token=AEEZVCVFILGIOJ66C4F77DLQXGUH7A5CNFSM4JRMCLDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGCXJOY#issuecomment-562394299, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEEZVCRBHCA4IWZCAN7FKO3QXGUH7ANCNFSM4JRMCLDA .
Yes. The drawing of the boxes is actually optional, afters it trained when you run the detection code it give you back a list/array of detections with the class, box coordinates and percentage of certainty. By default it draws every box but you could use this data to "classify" the image and draw boxes around some or all of the detections as you see fit.
I personally have modified the detection code to run on live camera footage, with frame skipping that draws the same box across multiple changing frames aka the footage stays real time while the detection only updates every few frames.
@NiklasWilson how did you add negative images in the data_train.txt file. I am looking for a solution for a long time but could not fine any. If I just try to give images name without class in the data_train.txt , it throws an error.
@shahzaibraza37 I check a folder for any image files not mentioned in the annotations file that gets generated. Any that are missing I add to the annotations file. (I think, just did a quick review of my code change for that)
Here is the actual commit, it only modifies two files so it shouldn't be too hard to find. It adds a '--no_object_folder' param to 'Convert_to_YOLO_format.py ' and it adds a 'zeroAnnotationPath' to 'Utils/Convert_Format.py'
The actual code change is in 'Utils/Convert_Format.py' which gets called from 'Convert_to_YOLO_format.py'
If you can't figure it out let me know maybe I can try to walk through it more. Assuming this link is even public. https://github.com/NiklasWilson/TrainYourOwnYOLO/commit/4d314d24d43865b9d377c00609c4945113695d84
Hi @NiklasWilson ,
So, I have changed the script as you mentioned and everything has worked for my data_train.txt file. The problem is same! When I try to train my model it shows the same error as before:
FileNotFoundError: [Errno 2] No such file or directory: 'Y:/955-030/MA_Hafiz/Machine_learning/Training_set/Train_14_04/TrainYourOwnYOLO/Data/Source_Images/no_object/P8 (67).png \n
All the images are there in the no_object folder. Do you have any solution for it? The error is appearing because there is no class and bounding box to these negative images. There should be a some simple way of using these negative images for training of the model.
Remove spaces from all your file names. Its literally saying that it is not seeing that file and in my experience it was related to spaces or slashes. different parts of the code handle the file path differently and some seem fine with it and others seems to break on it. (If you copied my code directly, it was only tested with files named a certain way) I just mass renamed every file.
How about this: Let’s agree on a tag for an object that will be ignored by the model. That tag could be NONE. It would be used like this: The depicted concrete pad gives me problems, the model thinks it’s a cat named MrKuro. To stop the model from doing so, I would add a NONE tag to VOTT, I would tag that concrete pad a few times with NONE and feed it into my training set along with all the other pictures. Once trained, the model would simply ignore, not report, and not box-in any detected NONE objects. This way, the NONE tag could be added to a variety of undesirable objects, no special folders would need to be created, and no workflow would be interrupted. The solution could be as simple as sticking a
if predicted_class != "NONE":
into the for i, c in reversed(list(enumerate(out_classes))): loop in yolo.py
Hi @bertelschmitt @NiklasWilson,
I did not solve the problem of negative images because of the error
FileNotFoundError: [Errno 2] No such file or directory: 'Y:/955-030/MA_Hafiz/Machine_learning/Training_set/Train_14_04/TrainYourOwnYOLO/Data/Source_Images/no_object/P8 (67).png \n
Even though, there was not a single space in the name of non_object training images as suggested by @NiklasWilson. What I did, I annotated the false results in the image as some other class and my results are getting better and better.
@bertelschmitt The type of solution you are suggesting will surely improve the results alot, because then there will be no extra/unnecessary classes.
@bertelschmitt My setup doesn't use a separate folder. (I pass the the folder with the annotation images as my no_object folder) It checks the annotation file and only adds the entries that are absent... the option for the folder to be different was more of a formality.
I strongly discourage using no object tags. It will result in decreased accuracy when your no_objects are not present. Using images with no_object tags is not the same as images with no annotations. You will be training it to detect a singular object as a no_object which will will be very inaccurate with the more unique things you page with it. It will also waste a lot of processing detecting these.
@shahzaibraza37
This P8 (67).png is your issue. The code that handles file paths does not like the special characters. It breaks on that, even though the error is using the full path the thing that tried to read the path didn't. (filenames are part of the path)
@NiklasWilson I meant to say, I changed the files name to e.g q.png but it still gave the same error as I told earlier.
look at the annotations file. I had this exact same issue originally. Its related to how the file paths are read from the annotations file. I don't remember the solution, you will need to find where that error is thrown, and add a bunch of print statement anywhere where a file path / filename is used to track dow the exact issue.
also make sure that file actually exists in both the annotations file and the folder....
Double check the exact path that the error spits out is correct, manually follow the folders and confirm that it really matches a real file.
@NiklasWilson, not trying to argue, just trying to understand. What I propose changes nothing to the training and detection, it simply adds a very trivial post-processing step: YOLO will dutifully try to detect all objects in its model, except that when it finds one with a name chosen by myself (I chose “NONE”) that detection simply will be thrown out, and not reported up the chain. I currently have 13 objects in my model, and I wouldn’t think twice about adding a few more. So why should the addition of a 14th object called “NONE” wreak havoc with precision and lead to a pile of wasteful processing cycles? Please explain to me in greater detail where I’m wrong.
In any case, my solution involves one small change in detect_image in yolo.py (this is where the bounding boxes are added):
if predicted_class == "NONE":
continue
That’s it.
It definitely is not a general solution, but it is one that could be helpful in certain narrow cases. After my model misfired depending on the position of the sun, and the shade projected by the house, I stuck the two lines into my code, I marked the offending parts of the image (in my case, a slab of concrete) as “NONE,” and peace ensued. The computer did not work harder, or less hard than before, but most of all, it works for me.
@bertelschmitt I did the same thing with my case. It works perfectly. Now, I am trying to include the negative images also to see the effect of the negative samples on the results.
@NiklasWilson Hey. I tried to train the model again with zero annotations again and this time the error looks like this:
File "E:\Train\TrainYourOwnYOLO\2_Training\src\keras_yolo3\yolo3\utils.py", line 68, in get_random_data line = line[1].split(" ") IndexError: list index out of range
Do you have any recommendations?
@bertelschmitt ok I rethought what your doing. The main issue would be if the no_object itself changes. Like I have 6 cameras on my house and a singular no object would not work for all 6 as that in it self would create false negatives but I guess if your no object was of a singular background it could work just fine. In this case I guess both methods would have the same outcome. The specific idea behind the more general zero_annotations is that it increases the accuracy of the already learned objects by effectively unlearning data/patterns that were never meant to be learned.
@shahzaibraza37 line[1] does not exist. So either "line" is null/undefined or it doesn't have 2 items in it. The .split(" ") Should work on an any string even if spaces are absent but it has to be at least an empty string. Look at what ever sets line. Word of advice (not trying to be insulting) adding print statements throughout can greatly help in debugging for instance print what line is pulling from than print it before and after the split statement. Anywhere were it might help give an idea of what is happening.
@NiklasWilson Thanks for the advice. I have solve this issue by changing some lines in the 2_Training\src\keras_yolo3\yolo3\utils.py . Now I can negative images in my training data., Thanks for the help :)
@shahzaibraza37 Awesome glad it worked out. So to clarify you got it working with images that have zero annotations? Have you noticed an improvement or are still working on actually getting to test it out?
@NiklasWilson I am still testing it. But from the initial results, it looks like it will not have a huge impact on the results. But I will share my thoughts on this method after doing thorough tests.
@shahzaibraza37 Ok. I think it really depends on the content. I like to test mine on video footage such as youtube videos. One I like watching it action and two this way I can see how it performs on data that I never considered on presenting it. I used the zero annotations to correct things it routinely thought matched. For instance when I was just playing around with it I make it detect weapons, it routinely thought dining chairs looked like handguns, after feeding it zero annotation images of dinging rooms that was no longer an issue.