About checkpoints and stable-diffusion-v1-5

[Rts-Wu]

Thanks for your wonderful work! And I have some questions:

1. The checkpoint link you provided includes the folders vae, scheduler, tokenizer, and unet. However, in the pretrained section, you mentioned downloading the pre-trained StableDiffusion-1.5 from SDM, which also includes vae, scheduler, tokenizer, and unet. Which files should I use during training?

2. In your ./data_process/README.md, it mentions needing to download the required checkpoints such as yolov7.pt. Where can I find these checkpoints?

[Rts-Wu]

And if I want to use the 'Testset' uploaded to 'HuggingFace', only extracting the keyframes and their corresponding subtitles via 'python ./data_process/extract.py' is enough? I mentioned you said in StorySalon Readme: use the provided data processing pipeline to obtain the frames. But in issues you said it had been processed.Could you show more details about how to use the dataset? Thx a lot!

[haoningwu3639]

1. Since our model is designed based on SDM, all SDM pre-trained parameters need to be used;
2. Please refer to the official implementation repository of yolov7: https://github.com/WongKinYiu/yolov7
3. We have processed this part of the test set and can use it directly.