ml-commons icon indicating copy to clipboard operation
ml-commons copied to clipboard

Support OCI connector and GenAI

Open khoaisohd opened this issue 1 year ago • 4 comments

Is your feature request related to a problem? We need to support a connector to the Oracle Cloud Infrastructure (OCI) and OCI GenAI service as ML so that the conversation search can work for Oracle Cloud Infrastructure

What solution would you like? From the customer, we are going to add new connector protocols and ML Provider

  • Add a new connector protocol oc1_sigv1 to allow ML common to talk to OCI services.
  • Add new ModelProvider oci_genai

Implementation changes

  • Add OCI common SDK dependencies for oc1_sigv1 protocol implementation
  • Add new logic to generate prompt and OCI GenAi request and parse response from OCI GenAi response

What alternatives have you considered? A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context? Example of OCI remote connector

{
     "name": "OpenAI Chat Connector",
     "description": "The connector to public OpenAI model service for GPT 3.5",
     "version": 2,
     "protocol": "oci_sigv1",
     "parameters": {
         "endpoint": "inference.generativeai.us-chicago-1.oci.oraclecloud.com",
         "auth_type": "resource_principal"
     },
     "credential": {
     },
     "actions": [
         {
             "action_type": "predict",
             "method": "POST",
             "url": "https://${parameters.endpoint}/20231130/actions/generateText",
             "request_body": "{\"prompts\": [\"${parameters.prompt}\"], \"maxTokens\": 300, \"temperature\": 1, \"frequencyPenalty\": 0, \"presencePenalty\": 0, \"topP\": 0.75, \"topK\": 0, \"compartmentId\": \"ocid1.compartment.oc1..aaaaaaaadhycphouqp6xuxrvjoq7czfv44o2r5aoz53g3bw6temcu432t76a\", \"returnLikelihoods\": \"NONE\", \"isStream\": false, \"stopSequences\": [], \"servingMode\": { \"modelId\": \"cohere.command\", \"servingType\": \"ON_DEMAND\" } }"
         }
     ]
 }

Search request

{
  "query": {
    "match": {
      "text": {
        "query": "Which team Phuong worked in Oracle"
      }
    }
  },
  "ext": {
    "generative_qa_parameters": {
      "llm_model": "oci_genai/cohere.command",
       "llm_question": "Which team Phuong worked in Oracle",
       "conversation_id": "T8DMa40BQ5L2Y1cB6PVX",
       "context_size": 5,
       "interaction_size": 1,
       "timeout": 15
      }
   }
}

khoaisohd avatar Feb 07 '24 21:02 khoaisohd

@ylwu-amzn @samuel-oci would it be possible to assign this issue to me?

khoaisohd avatar Feb 08 '24 04:02 khoaisohd

@ylwu-amzn @samuel-oci would it be possible to assign this issue to me?

yes, @ylwu-amzn for some reason I am still not able to assign issues, can you please check my permissions are in place?

sam-herman avatar Feb 08 '24 04:02 sam-herman

Sure, assigned to you @khoaisohd

Will take a look tomorrow @samuel-oci , maybe some configuraion missed

ylwu-amzn avatar Feb 08 '24 08:02 ylwu-amzn

I created this PR to address this issue https://github.com/opensearch-project/ml-commons/pull/2088 cc @samuel-oci @ylwu-amzn

khoaisohd avatar Feb 13 '24 18:02 khoaisohd