tlpss
Coco dataset , fix for grayscales images, convert them to RGB
@ExtReMLapin thanks for your contribution, looks like a useful addition! I'll merge it soon.
🖖🏻
A pleasure.
I also have more changes in staging for annother PR which adds max_image_size param
I'm training it on forensic images that have different resolutions and often high ones which causes :
- OOM during training (because of big resolution)
- Error during validation because of torch.stack trying to stack up different sizes.
Hi @ExtReMLapin
Sounds like an interesting project.
I consider these steps part of the preprocessing to reduce the burden on the ML codebase (can't support everything in the training loops) and to increase data loading speeds (loading a huge image from disk and then resizing it can bottleneck the GPU because it has to wait on the CPU, which is not desirable).
I will probably not accept a PR that does image resizing in the dataloader (as a separation of concerns).
You should consider resizing the images upfront into a separate dataset and only then training a detector on them.
I have some code for this here if you are interested.
@ExtReMLapin can you take a look at the CI failures? apparently one of the tests was broken by an update in torch ,but the fix should be straightforward.
Btw, I'm on a conference next week so will take some time for me to get back to you! But I do appreciate the PRs :slightly_smiling_face:
No worry with the delay.
To be frank i've been working on this forensic minutiae detector for two years and you have no idea how sometimes it's a pain in the ass to :
- set up the whole repository env
- transform your dataset
- discover their undocumented training examples are not working
here it's just working with wandb integration, few issues with DDP but it's fine tbf