Is there any support for visual generation?

[dunbar12138]

Hi, thanks for the great repo.

I'm wondering how I can use the Lit Autoregressive models in the image/video generation tasks.

Is there any doc / tutorial about how to prepare visual tokens as training set, and how to make the LitGPT/llama model take in tokens instead of text?

Or even better, is there any tutorial / example about how to integrate LitGPT/llama into a normal pytorch lightning training recipe (potentially with other models getting trained together), and customize the architecture and training instead of being a standalone model itself?

[rasbt]

Hi there!

Image/video is not supported but I can surely add a Trainer recipe some time. Thanks for suggesting!

[dunbar12138]

Thanks for the reply! That would be great. Let me know when the tutorial is added. I really appreciate it!