Feat: semi self-hosted LLMs using AzureML Online Endpoints

[janaka]

Description

Implements #10

Enables deploying an OSS LLM like Llama2 on AzureML + LlamaIndex level API implementation.

• [x] IaC for infra using ARM templates
• [x] AzureML Online Endpoints Web API client
• [x] LlamaIndex LLM class for AzureML self-hosted models
• [ ] Test
• [ ] maybe some LLM class code folder re-organisation

Note:

• Docq application-level integration is not part of this PR and depends on #12.

Type of change

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Refactor and code improvement (non-breaking change which improves code quality and/or performance)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] Documentation
• [ ] Tests
• [ ] Other chores such as maintenance

How Has This Been Tested?

Current not tested

• [ ] Test A
• [ ] Test B
• [ ] Test C

Test Configuration:

There two methods for setting up the Model backend for testing

• Spin up an Online Endpoint with Llama2-7b-chat on Azure
  • pro: this is a matter of running the ARM template via the shell script.
  • con: slow to spin up/down the infra 15-20mins. the infra is expensive ~$150 /day
• Spin up a local Online Endpoint with Llama2-7b-chat on Azure
  • pro: zero cloud bill cost.
  • con: speed of spin up is not known for sure but assume still slow. some work to figure out local Online Endpoints

Checklist:

• [ ] My code follows the style guidelines of this project
• [ ] I have performed a self-review of my own code
• [ ] I have commented my code in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] My changes generate no new warnings
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] New and existing unit tests pass locally with my changes
• [ ] Any dependent changes have been merged and published in downstream modules
• [ ] I have checked my code and corrected any misspellings
• [ ] The commit message follows the convention of this project

[cwang]

Nice, a few comments:

• If any of the files are exported instead of drafted from scratch, would be nice to state how they are created in Azure console and how they are exported.
• Docs should probably go into the doc site instead of staying with the README file here.

[janaka]

Nice, a few comments:

• If any of the files are exported instead of drafted from scratch, would be nice to state how they are created in Azure console and how they are exported.

Agreed - though not expecting exported files. There are a bunch of files that aren't needed which needs to be cleaned up.

• Docs should probably go into the doc site instead of staying with the README file here.

Yep