KwaiVGI
Training of the second stage
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
Hi,May i ask how did you train this model. Did you write the training code and collect train-data yourself?
Hi,May i ask how did you train this model. Did you write the training code and collect train-data yourself?
Yes, we have implemented training codes and collected datasets by ourselves.
Hi @JZArray, can you share the training code and sample dataset that needs for training
@JZArray Hi, I want to ask do you know how to train on images? the paper says they train the model on both images and videos, but image just has one frame, so we just set source image and target image the same? Then will the model learn something useful because it can just copy and paste the image?
@JZArray Hi, I want to ask do you know how to train on images? the paper says they train the model on both images and videos, but image just has one frame, so we just set source image and target image the same? Then will the model learn something useful because it can just copy and paste the image?
Same ID but different two frames as source and target respectively
Hi @JZArray, can you share the training code and sample dataset that needs for training
Sry, I can't upload my codes and datasets...
Hi,May i ask how did you train this model. Did you write the training code and collect train-data yourself?
Yes, we have implemented training codes and collected datasets by ourselves.
how can you to get sholder mask? and can you can considerable result as author pretrained model?
Hi @JZArray, can you share the training code and sample dataset that needs for training
Sry, I can't upload my codes and datasets...
Hi,May i ask how did you train this model. Did you write the training code ？
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
have you save the problem, if yes, can you introduce some suggestion about training second stage
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
I did it this way, but the fine-tuning results were terrible.
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
I did it this way, but the fine-tuning results were terrible.
Do you follow this codebase? https://github.com/zhanglonghao1992/One-Shot_Free-View_Neural_Talking_Head_Synthesis
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
I did it this way, but the fine-tuning results were terrible.
Do you follow this codebase? https://github.com/zhanglonghao1992/One-Shot_Free-View_Neural_Talking_Head_Synthesis
Yes, and I used the MEAD and VFHQ datasets.
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Nope
Hi @JZArray, can you share the training code and sample dataset that needs for training
Sry, I can't upload my codes and datasets...
Hi,May i ask how did you train this model. Did you write the training code ？
Yes, we write our own training codes @lmpeng12
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Nope
Do you have any suggestions on crop method? Thank you very much.
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Nope
Do you have any suggestions on crop method? Thank you very much.
You can use any detection method to first ensure a person's position (face position), then fix the bbox to do crop for all frames
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
I did it this way, but the fine-tuning results were terrible.
Do you follow this codebase? https://github.com/zhanglonghao1992/One-Shot_Free-View_Neural_Talking_Head_Synthesis
Yes, and I used the MEAD and VFHQ datasets.
Could you please show some samples of your model? Thanks
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
I did it this way, but the fine-tuning results were terrible.
Do you follow this codebase? https://github.com/zhanglonghao1992/One-Shot_Free-View_Neural_Talking_Head_Synthesis
Yes, and I used the MEAD and VFHQ datasets.
Could you please show some samples of your model? Thanks
Sry, I can't upload them
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Nope
Do you have any suggestions on crop method? Thank you very much.
You can use any detection method to first ensure a person's position (face position), then fix the bbox to do crop for all frames
If the bbox is fixed based on the first frame, is it possible that the head could move outside the bbox in subsequent frames？
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Nope
Do you have any suggestions on crop method? Thank you very much.
You can use any detection method to first ensure a person's position (face position), then fix the bbox to do crop for all frames
hi, did you add Region Loss and Wing Loss, and which ten points did you choose?
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Nope
Do you have any suggestions on crop method? Thank you very much.
You can use any detection method to first ensure a person's position (face position), then fix the bbox to do crop for all frames
If the bbox is fixed based on the first frame, is it possible that the head could move outside the bbox in subsequent frames？
yes, it is possible, so you need to ensure there are no large head motions
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Nope
Do you have any suggestions on crop method? Thank you very much.
You can use any detection method to first ensure a person's position (face position), then fix the bbox to do crop for all frames
hi, did you add Region Loss and Wing Loss, and which ten points did you choose?
yes, there is a issue asking the same question, and author explained it, you can have a check
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Nope
Do you have any suggestions on crop method? Thank you very much.
You can use any detection method to first ensure a person's position (face position), then fix the bbox to do crop for all frames
hi, did you add Region Loss and Wing Loss, and which ten points did you choose?
yes, there is a issue asking the same question, and author explained it, you can have a check
Thank you. Can you reveal how many ID videos were used?
Hello, a quick question, I am trying to train this model from scratch, and it seems the first-stage training is ok, and I have gotten a promising result. While training the second stage, results are always noised, so I want to ask how to train the second one? Should a pretrained model from the first stage be frozen, and only the stitching module is going to be trained? Whether only using stitching loss to supervise stitching module is enough, or losses used in the first stage should be also used?
hi, How did you create the dataset？use ”crop_driving_video()“ for a video or "crop_source_image" for each frame。
Hi, i have the same question too. Have you saved the problem?
I have the same problem. I am not sure but I think they use ”crop_driving_video()“ for a video.
Nope
Do you have any suggestions on crop method? Thank you very much.
You can use any detection method to first ensure a person's position (face position), then fix the bbox to do crop for all frames
Sorry, I don't quite understand the purpose of doing this kind of crop. Are you trying to align the first frame with the crop_source_image? In that case, during training, can we only use the first frame of each video as the source?