add databricks provider

[prithu-dasgupta]

We want to setup Databricks as a remote inference provider for llama-stack. Databricks Foundation Model APIs are OpenAI compatible and we suggest using the OpenAI client to query Databricks model serving endpoints.

How I tested

LLAMA_STACK_DIR=. PYTHONPATH=. python -m llama_stack.cli.llama stack build (set name as databricks-test and provider as remote::databricks and )

LLAMA_STACK_DIR=. PYTHONPATH=. python -m llama_stack.cli.llama stack configure databricks-test (set databricks workspace url, databricks workspace api token, Llama3.1-70B-Instruct as supported model, other fields as default)

LLAMA_STACK_DIR=. PYTHONPATH=. python -m llama_stack.cli.llama stack run databricks-test --port 12345

Query the endpoint

> llama-stack % curl -X POST http://localhost:12345/inference/chat_completion -H "Content-Type: application/json" -d '{"model":"Llama3.1-70B-Instruct","messages":[{"content":"hello world, write me a 2 sentence poem about the moon", "role": "user"}],"stream":false}'
data: {"completion_message":{"role":"assistant","content":"Silvery light upon my face, \nThe moon's soft glow, a peaceful place.","stop_reason":"end_of_turn","tool_calls":[]},"logprobs":null}

> % curl -X POST http://localhost:12345/inference/chat_completion -H "Content-Type: application/json" -d '{"model":"Llama3.1-70B-Instruct","messages":[{"content":"hello world, write me a 2 sentence poem about the moon", "role": "user"}],"stream":true}' 
data: {"event":{"event_type":"start","delta":"","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":"Sil","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":"very","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" moon","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":",","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" a","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" glowing","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" sphere","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":",\n","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":"D","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":"ancing","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" in","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" the","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" darkness","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":",","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" calm","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" and","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":" clear","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"progress","delta":".","logprobs":null,"stop_reason":null}}

data: {"event":{"event_type":"complete","delta":"","logprobs":null,"stop_reason":"end_of_turn"}}

(venv) prithu.dasgupta@HPW57GX3RK llama-stack %