How can I run transform program smoothly?

[yuya-0411]

Hello.

I could make the environment for this repository and tried running transform program. But I got documentation for Memory Management and PYTORCH_CUDA_ALLOC_CON as an error. That seems a lack of gpu capacity, so I tried it another PC with RTX2080. But I got the same error.

When I stopped all programs except for style_transfer.py, I got the same error.

Do you have any ideas to solve this problem?

Thank you for your attention.

[williamyang1991]

I think you can set batch_size to a small value like 1 to see if the memory is still not enough. --batch_size 1

https://github.com/williamyang1991/VToonify/blob/920b56478835873169b31dd3d134d29e7e16f94b/style_transfer.py#L35

[yuya-0411]

@williamyang1991 Thank you for your reply. I could transform sample video using --batch 1. But sometimes I get an error as shown below.

What does it mean? How should I solve this? What's more, I always get this error when I use my own video. Do you have any standards for input video?

Thank you for your attention.

[williamyang1991]

I have also enountered this issue previously. This happens when there is no face detected in your video. I think this is a bug of the dlib.shape_predictor. Most of time, it can detect the face in 529.mp4, but sometimes it fails to do so. (very strange, the same video, but different results) You can just rerun it for some times, until it succesfully detect the face.

[yuya-0411]

@williamyang1991 Ok. Thank you for your time and your polite suggestion. I will try it.

[williamyang1991]

You are welcome.

I have also enountered this issue previously. This happens when there is no face detected in your video. I think this is a bug of the dlib.shape_predictor. Most of time, it can detect the face in 529.mp4, but sometimes it fails to do so. (very strange, the same video, but different results) You can just rerun it for some times, until it succesfully detect the face.

Insterestingly, I only got this issue twice on the Colab. When I run the code on my own machine, I never enounter this issue.

[yuya-0411]

@williamyang1991 Really!?

I got the error, running on my local environment... It's weird though. I tried continuing running more than 30 times, but I can't get transformed media.

I will try code on Colab as well.

thank you

[williamyang1991]

Yes, quite weird. This week I have run many results with the code on my local environment without this issue. I only got the error on Colab.

[williamyang1991]

I'm very confused. This face detection code is directly copied from DualStyleGAN, where no one report this issue.

https://github.com/williamyang1991/DualStyleGAN/blob/25f86a445362dd7bdf2ad4391afbd9dca162e9c1/model/encoder/align_all_parallel.py#L32-L49

[LinearVelocity]

@yuya-0411 I've encountered the same issue as you. As illustrated in the code (style_transfer.py), for each video, the face in the first frame is detected and aligned for pSp to obtain the style code, which means you must ensure there are faces in the first frame in your input videos. Owing to the face detector is dlib.shape_predictor, front faces make it easier to succeed.

[yuya-0411]

@LinearVelocity Thank you for your information. I also succeeded by ensuring there were faces in the first frame. I removed some frames for video data whose first frame has no face.