pytorch-CycleGAN-and-pix2pix

pytorch-CycleGAN-and-pix2pix copied to clipboard

I have paired IR and RGB traffic images (video sequences). My goal is to achieve IR colorization at night. The effect is the same as that of RGB in the daytime. I have tried many methods, such as cyclegan, pix2pix, pix2pixHD, vid2vid, few shot vid2vid, etc.

Since there are many traffic scenario, the traffic scenarios in different areas are different, and my dataset scenarios are limited, I would like to ask which method is the most appropriate and has the best generalization ability? Among the results of my current attempts, pix2pixHD works well, but after changing the scene, the effect is not good. Do you have any reasonable suggestions? Thank you very much for your reply！！

Most of the methods tend to overfit the training set if you have limited data. You may consider applying data augmentation to combat model overfitting.

@junyanz So which method (including but not limited to) do you think is more suitable for the scenario I described？

It may not be so related to the models you are using. You need to either experiment with different types of data augmentation or expand your dataset.

At present, I want to color infrared video. Is there any good way?

You can try fewshot vid2vid. If you have a smaller dataset, you need to reduce the network's capacity as well. It's hard to make any above methods work out of the box.