Lightning-AI
Add deploy example
The tricky part adding a simple deploy example is that
- the outputs are printed inline.
- Also, each time someone makes a request the model is loaded into memory, which is not very efficient.
Maybe the best way to address both points above is to define a new convenience function to LitGPT that keeps the model loaded like in chat but then also yields the text outputs generated by the model. Or something like that.
Any ideas or suggestions here @awaelchli and @carmocca ?
Fixes #1259
This is awesome, Carlos, and it works great! I updated the Readme and added a tutorial.
A little note about the structure: As far as I understand, this was requested to be a subsection in the Get Started main section. I have the full example in the deploy.md document and also linked it in the Zero to LitGPT guide because the Zero to LitGPT guide is referred to as the one "See the full docs" example. I think this is probably the structure we have to go with.