llama-recipes icon indicating copy to clipboard operation
llama-recipes copied to clipboard

[OctoAI] Introducing a Video Generation use case featuring Llama3

Open tmoreau89 opened this issue 2 months ago • 3 comments

What does this PR do?

This example under lama-recipes/recipes/use_cases demonstrates a "model cocktail" of Llama3, image generation and image animation to produce a full recipe video based on a simple text prompt - the name of the dish.

In particular this demo shows how one can use Llama3's JSON formatting capabilities to generate a scene by scene description of a recipe making video.

This demo is powered by OctoAI as the model API provider but the user can feel free to choose whichever model API provider they'd like.

Feature/Issue validation/testing

This demo was run on Jupyter notebook on a Macbook M1 machine and in Google Colab (T4-based).

Before submitting

  • [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • [x] Did you read the contributor guideline, Pull Request section?
  • [ ] Was this discussed/approved via a Github issue? Please add a link to it if that's the case.
  • [x] Did you make sure to update the documentation with your changes?
  • [ ] Did you write any new necessary tests?

Thanks for contributing 🎉!

tmoreau89 avatar May 11 '24 01:05 tmoreau89