Open-Sora-Plan

Open-Sora-Plan copied to clipboard

Some question about resume training causalvae

1

When I want to resume train causalvae in our own dataset with following script, it always report the following bug ```shell _pickle.UnpicklingError: invalid load key, '\xbb'. ``` here are my...

ZhikangNiu

Questions about the value range of input to CausalVAE

1

In video dataset: https://github.com/PKU-YuanGroup/Open-Sora-Plan/blob/098ecbbd5d7ecb85d8e38e92edfc9466b4c65f3a/opensora/models/ae/videobase/dataset_videobase.py#L50 says [-0.5, 0.5] while the transform sounds like map data to [-1, 1] https://github.com/PKU-YuanGroup/Open-Sora-Plan/blob/098ecbbd5d7ecb85d8e38e92edfc9466b4c65f3a/opensora/models/ae/videobase/dataset_videobase.py#L66 So what's the value range of input to CausalVAE? Many thanks!

longyangqi

+ 整理了sample_t2v.py，方便部署。 + 针对sample的内容添加了一些type hint 在我的服务器上能正常推理

FEIFEIEIAr

hw910B NPU显卡t2v效果复现问题

9

你好！感谢你们为文生视频开源工作的付出！目前我在华为 910B显卡上运行`scripts/text_condition/sample_video.sh`的推理工作，按`examples/prompt_list_0.txt`的prompt list上运行推理并复现效果，主要存在以下问题： （1）17x256x256的生成较为正常，可复现结果。 （2）65x256x256随机存在部分Prompt（约2/16）的视频生成失败（即生成单调颜色），且失败的prompt随随机种子的变化而变化。 （3）65x512x512存在更多Prompt（约3/16-4/16）的视频生成失败，且成功的部分视频patch之间的间隙较为明显。 请问大概是什么原因？是权重加载未完全的原因，还是代码框架与NPU底层架构的适应性问题？期待与您的讨论！

ljrprocc

Thank to your advice, I could finetune your model on Zudamon Anime dataset. (I can use authorized Zudamon Anime dataset.) https://github.com/PKU-YuanGroup/Open-Sora-Plan/assets/3625196/a76c1ebc-96ab-4b65-9f9b-7469b158d432 But, I could do video captioning few videos manually...

alfredplpl

作者你好，请问执行pip install -e ".[train]"报错如下图所示是啥原因

2


cfy3133

Hello, I have noticed the install instructions specify a python 3.8 requirement. What would it take for OSP to support 3.11? I get an issues running model when using 3.11...

kernelzeroday

请问你们的数据集下载后，怎么解压呢？

2

ersanliqiao

RuntimeError: dim ([32]) cannot convert to NDC1HWC0

1

**Problem:** Encountered **RuntimeError: dim ([32]) cannot convert to NDC1HWC0** when trying to reconstruct a video using **rec_imvi_vae.py** **Command:** `python examples/rec_imvi_vae.py --model_path /LanguageBind/Open-Sora-Plan-v1.0.0 --video_path test_video.mp4 --rec_path rec_video.mp4 --fps 24 --resolution 512...

amoybot

Inquiry of training time for CausalVideoVAE

1

Thanks for making the amazing work publicly available. I am curious about the total training time for your pre-trained VAE model: https://huggingface.co/LanguageBind/Open-Sora-Plan-v1.0.0/tree/main/vae How long have you spent to train this...

FlyingRoastDuck