INFRA: Self-hosted Models

[cwang]

Is your feature request related to a problem? Please describe.

This would solve one problem:

• dependency on cloud-vendor-hosted models (Azure OpenAI and GCP PaLM) And open up one opportunity:
• future fine-tuning models under-the-hood

Describe the solution you'd like

Very much an infra challenge. Incorporating it into existing provisioning (ARM for Azure, CF for AWS) would be the first step.

Describe alternatives you've considered

No alternative.

Additional context

Likely to start with models available and popular on HuggingFace, with commercial-friendly licenses such as ASL2 (Falcon, OpenAssistant and Dolly). See https://github.com/eugeneyan/open-llms