Open-Sora icon indicating copy to clipboard operation
Open-Sora copied to clipboard
verified publisher icon hpcaitech

About the 4x32x32 AE and DiT checkpoint

Open jiujing23333 opened this issue 9 months ago • 5 comments

Thank you very much for the impressive 2.0 version! After reading the README, I have a few questions. I noticed that the 4x32x32 compressed AE and DiT models are not available for download. Could you please let me know when they will be accessible? Additionally, could you provide more details about the currently available Hunyuan VAE-trained models? For example, sample outputs, quality metrics, and comparisons with the full Hunyuan T2V model would be helpful.

jiujing23333 avatar Mar 13 '25 12:03 jiujing23333

Sorry for late reply, you could check some details here: https://github.com/hpcaitech/Open-Sora/blob/main/docs/hcae.md

As for sample outputs, you could refer to our gallery: https://hpcaitech.github.io/Open-Sora/

quality metrics you could see our win-rate score and vbench results here: https://github.com/hpcaitech/Open-Sora?tab=readme-ov-file#evaluation

SimonWXW avatar Mar 14 '25 06:03 SimonWXW

Sorry for late reply, you could check some details here: https://github.com/hpcaitech/Open-Sora/blob/main/docs/hcae.md

As for sample outputs, you could refer to our gallery: https://hpcaitech.github.io/Open-Sora/

quality metrics you could see our win-rate score and vbench results here: https://github.com/hpcaitech/Open-Sora?tab=readme-ov-file#evaluation

Thanks for reply. From what I understand, the currently released model is a version based on ​HunyuanVAE + MMDiT. My confusion lies in whether the models used for comparisons in the technical report and for generating the Gallery examples are based on ​HunyuanVAE or ​HighCompress VAE?

jiujing23333 avatar Mar 14 '25 08:03 jiujing23333

Hi, we forgot to provide the download link previously, it is now updated in https://github.com/hpcaitech/Open-Sora/blob/main/docs/hcae.md.

As for the comparison and gallery, we use the HunyuanVAE + MMDiT

SimonWXW avatar Mar 14 '25 10:03 SimonWXW

Hi, we forgot to provide the download link previously, it is now updated in https://github.com/hpcaitech/Open-Sora/blob/main/docs/hcae.md.

As for the comparison and gallery, we use the HunyuanVAE + MMDiT

Will the HighCompress VAE + MMMDiT described in the tech report be open source?

sijeh avatar Mar 15 '25 02:03 sijeh

Hi, we forgot to provide the download link previously, it is now updated in https://github.com/hpcaitech/Open-Sora/blob/main/docs/hcae.md. As for the comparison and gallery, we use the HunyuanVAE + MMDiT

Will the HighCompress VAE + MMMDiT described in the tech report be open source?

yes, it is already opensourced, you can check the details in https://github.com/hpcaitech/Open-Sora/blob/main/docs/hcae.md.

SimonWXW avatar Mar 17 '25 01:03 SimonWXW