accelerated-data-science

accelerated-data-science copied to clipboard

Important

This PR is intended solely for demonstration purposes and is not meant to be merged.

Description

This PR includes handlers that were developed for the original Playground. It provides support for managing multiple sessions and threads, and also maintains a history of interactions with the model. It utilizes the SQL Alchemy to store the messages and threads.

Additional Packages

• sqlalchemy>=2.0.25

How To Use

To test handlers you will need to run ADS and JupyterLab. Add the extension to Jupyter Lab: jupyter serverextension enable --py ads.aqua.extension Start Jupyter Lab jupyter lab or jupyter lab --no-browser.

CLI

Add new playground session

ads aqua playground session create --model_id ocid1.datasciencemodeldeployment.oc1.iad....

Get sessions list

ads aqua playground session list

Get session details

By model ID (Model Deployment)

ads aqua playground session get --id ocid1.datasciencemodeldeployment.oc1.iad.....

By session ID

ads aqua playground session get --id 1

Handlers

Create Session

POST REQUEST

http://localhost:8888/aqua/playground/session

DATA

{
  "model_id": "ocid1.datasciencemodeldeployment.oc1.iad...."
}

Create New Thread

POST REQUEST

http://localhost:8888/aqua/playground/thread

DATA

{
  "messageId": null,
  "parentMessageId": null,
  "sessionId": 1,
  "content": "Who was the third president of the US?",
  "model_params": {
    "maxTokens": 500,
    "temperature": 0.5,
    "topK": 10,
    "topP": 0.5
  }
}

Get Thread Details

GET

http://localhost:9999/aqua/playground/thread/1

RESPONSE

{
  "id": 1,
  "name": "Who was the first president of the US?",
  "session_id": 1,
  "created": "2024-01-22 14:20:14.947342",
  "status": "active",
  "messages": [
    {
      "message_id": 1,
      "parent_message_id": null,
      "session_id": 1,
      "thread_id": 1,
      "content": "Who was the first president of the US?",
      "payload": {},
      "model_params": {
        "model": "/opt/ds/model/deployed_model",
        "max_tokens": 500,
        "temperature": 0.5,
        "top_p": 0.5,
        "frequency_penalty": 0.0,
        "presence_penalty": 0.0,
        "top_k": 10,
        "echo": false,
        "logprobs": null,
        "use_beam_search": false,
        "ignore_eos": false,
        "n": 1,
        "best_of": 1,
        "stop": null,
        "stream": false,
        "min_p": 0.0
      },
      "status": "active",
      "rate": 0,
      "role": "user",
      "created": "2024-01-22 14:20:17.639226"
    },
    {
      "message_id": 2,
      "parent_message_id": 1,
      "session_id": 1,
      "thread_id": 1,
      "content": "\n\nThe first president of the United States was George Washington. He was inaugurated on April 30, 1789, and served two terms in office until March 4, 1797.",
      "payload": {},
      "model_params": {
        "model": "/opt/ds/model/deployed_model",
        "max_tokens": 500,
        "temperature": 0.5,
        "top_p": 0.5,
        "frequency_penalty": 0.0,
        "presence_penalty": 0.0,
        "top_k": 10,
        "echo": false,
        "logprobs": null,
        "use_beam_search": false,
        "ignore_eos": false,
        "n": 1,
        "best_of": 1,
        "stop": null,
        "stream": false,
        "min_p": 0.0
      },
      "status": "active",
      "rate": 0,
      "role": "system",
      "created": "2024-01-22 14:20:17.644696"
    },
]
}

Get session details

GET

http://localhost:8888/aqua/playground/session/1

RESPONSE

{
  "session_id": 1,
  "name": "LLama2-13b model deployment - vllm-openai - A10",
  "url": "https://modeldeployment.us-ashburn-1.oci.customer-oci.com/ocid1.datasciencemodeldeployment.oc1.iad....",
  "model_id": "ocid1.datasciencemodeldeployment.oc1.iad.....",
  "created": "2024-01-22 13:39:43.257680",
  "status": "active",
  "threads": [
    {
      "id": 1,
      "name": "Who was the first president of the US?",
      "session_id": 1,
      "created": "2024-01-22 14:20:14.947342",
      "status": "active",
      "messages": []
    },
    {
      "id": 2,
      "name": "Who is the James Bond??",
      "session_id": 1,
      "created": "2024-01-22 17:35:32.270386",
      "status": "active",
      "messages": []
    }
  ]
}

There's much interest in the idea of an ide to develop/test system/role prompts for different vertical applications. Prior to resorting to fine tuning

📌 Cov diff with main:

📌 Overall coverage: