Open-Sora icon indicating copy to clipboard operation
Open-Sora copied to clipboard

Published 20 hours ago • verified publisher icon hpcaitech

Trade off between VQ-VAEs and VAEs

Open Red-Fairy opened this issue 11 months ago • 2 comments

Dear authors,

Thanks for the great work!

While OpenAI's Sora itself and sota 3D VAEs (MAGVIT) uses the Vector Quantized (VQ-VAE) variant, open-sora adopts the KL-VAE. What's the rationale behind it and what could be the potential trade-off? Do you have any insight regarding this issue?

Thanks in advance!

Red-Fairy avatar Mar 20 '24 02:03 Red-Fairy