why taking so large memory

[frabob2017]

Hello, I am training around 70,000 images (on disk, it is around 2.7G) in google colab pro. But when I read it in YoloV5 and cache it, around 1/3 total images, 23251 images, it will take 18.3G RAM. May I ask if there is parameters to adjust in configure fires to reduce such memory use?

Although I can subdivide the data and train them separately, I want to put 1 large data and do training. I am still learning YoloV5.

AMP: checks passed ✅ optimizer: SGD(lr=0.01) with parameter groups 79 weight(decay=0.0), 82 weight(decay=0.002), 82 bias albumentations: Blur(p=0.01, blur_limit=(3, 7)), MedianBlur(p=0.01, blur_limit=(3, 7)), ToGray(p=0.01), CLAHE(p=0.01, clip_limit=(1, 4.0), tile_grid_size=(8, 8)) train: Scanning '/content/drive/MyDrive/RSNA_Cervical_Spine/boundingBox/RSNA_data/train/labels.cache' images and labels... 1941 found, 21310 missing, 0 empty, 0 corrupt: 100% 23251/23251 [00:00<?, ?it/s] train: Caching images (18.3GB ram): 100% 23251/23251 [00:51<00:00, 454.94it/s]

[glenn-jocher]

@frabob2017 This is very basic: images are normally compressed when saved on disk.

[frabob2017]

@frabob2017 This is very basic: images are normally compressed when saved on disk.

I though there is a configure file that can control the data format such as reducing from float64 to float32. something like that. I will try cache disk to see.

[glenn-jocher]

@frabob2017 images are loaded in uint8, there is nothing smaller.

[github-actions[bot]]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!