No detection of Person, every other class is fine

[freund-masasana]

Search before asking

• [X] I have searched the YOLOv8 issues and found no similar bug report.

YOLOv8 Component

Train, Val, Predict

Bug

So I have a weird one here.. I start my training from the pretrained yolov8x.pt. The class Person is fine at this point. now I start the fine-tuning and with each epoch the class Person gets worse and worse to a point where after 200 epochs no person will be detected. All other classes are fine and detecting as expected. Persons are only detected when rotated (which we found out by having one randomly rotated image in the dataset). There is another case where bicycles get detected as Person. They should be a different class Vehicle, but I suspect this is a different issue. I checked the translation script and all bicycles are converted to vehicle, but maybe I need to check the raw data again for this.

I checked the labels.jpg (attached) for the train data and it seems Person is not corrupt as they are by far the class with the highest attendance (as expected).

I also started a 1 epoch training with validation data as train data to get the same graphics and they seem fine also.

After 1 epoch of training on the actual train data with the actual validation data, this person hits pretty good results

But after 5 epochs of training, the model doesn't recognise the person that well anymore

Unless...

I checked the label files and the images. Everything seems ok and even in the preview batch images, persons are labeled correctly. There is almost no rotated pictures in the whole dataset (I found 1), so where is the problem? I don't know if this is actually a YOLO problem or not but I am all out of ideas. So any help is appreciated! I can also provide more info if needed.

Cheers

Environment

Ultralytics YOLOv8.0.126 🚀 Python-3.10.12 torch-2.0.1+cu117 CUDA:0 (NVIDIA GeForce RTX 3080 Ti, 12035MiB) Setup complete ✅ (12 CPUs, 31.2 GB RAM, 1766.8/1785.0 GB disk)

Minimal Reproducible Example

No response

Additional

The checks are from my machine. The main bulk of training were done on a cloud machine that is shut down now. But since the examples are generated on my machine and the problems persisted throughout, I thought it is ok to do so.

Are you willing to submit a PR?

• [ ] Yes I'd like to help by submitting a PR!

[github-actions[bot]]

👋 Hello @freund-masasana, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

Hey there 👋,

Thanks for the detailed explanation of your issue! It certainly sounds puzzling but let's see if we can help you troubleshoot this. From your description, it sounds like the model's ability to detect the "Person" class is degrading over epochs during fine-tuning, which is indeed unusual. Here are a few things you might consider:

1. Overfitting: It's possible that the model might be overfitting on non-person classes or specific kinds of backgrounds/contexts which do not include persons as frequently. Although it seems counterintuitive as "Person" is the most attended class, overfitting can sometimes manifest in unexpected ways.

2. Learning Rate Decay: If the learning rate is not being reduced over epochs, the model might be "forgetting" what it learned about the "Person" class. Ensure you're employing a learning rate scheduler that reduces the rate over time.

3. Data Augmentation: Since the issue seems to be mitigated with rotated images, it might help to introduce more varied data augmentation strategies, especially focusing on augmentations that modify object orientation and scale.

4. Class Weight Balancing: Though "Person" is the most present class, it might still be beneficial to experiment with class weighting to see if emphasizing the "Person" class more prominently during training helps.

5. Review Data Labels: Though you've checked, it might be worth revisiting the labeling to ensure no systemic biases or errors, particularly focusing on whether "Person" instances in various orientations are consistently well-labeled.

If you haven't already, you might also explore training with a different base model or adjusting the architecture to see if a particular configuration is more susceptible to this issue.

Here's a quick example of how you might introduce more aggressive data augmentation in your training setup:

# Add data augmentation
augmentations = {
    'rotation': 15,  # degrees
    'translate': 0.1,  # fraction of total image width/height
    'scale': 0.1,  # scale augmentations
    'shear': 0  # shear augmentation
}

model.train(data='coco128.yaml', epochs=100, imgsz=640, augmentations=augmentations)

Keep in mind, the specifics will depend on your current setup and how you're invoking the training process.

Don't hesitate to share more details or further observations. We're here to help 🚀.

Best regards!

[github-actions[bot]]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐