ml-commons
ml-commons copied to clipboard
Support OCI connector and GenAI
Is your feature request related to a problem? We need to support a connector to the Oracle Cloud Infrastructure (OCI) and OCI GenAI service as ML so that the conversation search can work for Oracle Cloud Infrastructure
What solution would you like? From the customer, we are going to add new connector protocols and ML Provider
- Add a new connector protocol oc1_sigv1 to allow ML common to talk to OCI services.
- Add new ModelProvider oci_genai
Implementation changes
- Add OCI common SDK dependencies for oc1_sigv1 protocol implementation
- Add new logic to generate prompt and OCI GenAi request and parse response from OCI GenAi response
What alternatives have you considered? A clear and concise description of any alternative solutions or features you've considered.
Do you have any additional context? Example of OCI remote connector
{
"name": "OpenAI Chat Connector",
"description": "The connector to public OpenAI model service for GPT 3.5",
"version": 2,
"protocol": "oci_sigv1",
"parameters": {
"endpoint": "inference.generativeai.us-chicago-1.oci.oraclecloud.com",
"auth_type": "resource_principal"
},
"credential": {
},
"actions": [
{
"action_type": "predict",
"method": "POST",
"url": "https://${parameters.endpoint}/20231130/actions/generateText",
"request_body": "{\"prompts\": [\"${parameters.prompt}\"], \"maxTokens\": 300, \"temperature\": 1, \"frequencyPenalty\": 0, \"presencePenalty\": 0, \"topP\": 0.75, \"topK\": 0, \"compartmentId\": \"ocid1.compartment.oc1..aaaaaaaadhycphouqp6xuxrvjoq7czfv44o2r5aoz53g3bw6temcu432t76a\", \"returnLikelihoods\": \"NONE\", \"isStream\": false, \"stopSequences\": [], \"servingMode\": { \"modelId\": \"cohere.command\", \"servingType\": \"ON_DEMAND\" } }"
}
]
}
Search request
{
"query": {
"match": {
"text": {
"query": "Which team Phuong worked in Oracle"
}
}
},
"ext": {
"generative_qa_parameters": {
"llm_model": "oci_genai/cohere.command",
"llm_question": "Which team Phuong worked in Oracle",
"conversation_id": "T8DMa40BQ5L2Y1cB6PVX",
"context_size": 5,
"interaction_size": 1,
"timeout": 15
}
}
}
@ylwu-amzn @samuel-oci would it be possible to assign this issue to me?
@ylwu-amzn @samuel-oci would it be possible to assign this issue to me?
yes, @ylwu-amzn for some reason I am still not able to assign issues, can you please check my permissions are in place?
Sure, assigned to you @khoaisohd
Will take a look tomorrow @samuel-oci , maybe some configuraion missed
I created this PR to address this issue https://github.com/opensearch-project/ml-commons/pull/2088 cc @samuel-oci @ylwu-amzn