jupyter-ai icon indicating copy to clipboard operation
jupyter-ai copied to clipboard

Allow additional properties in AgentChatMessage

Open andrii-i opened this issue 1 year ago • 0 comments
trafficstars

Problem

AgentChatMessage represents replies of LLM agents to users. Currently model is limited in its ability to handle diverse types of responses beyond plain text, for example error messages (see https://github.com/jupyterlab/jupyter-ai/pull/513/commits/b7ef4e32bf30932129444770b5872e36a8c19b35 in #513) or multi-modal responses that might include images or video. https://github.com/jupyterlab/jupyter-ai/blob/976f8b9303d198fb339f7b594d29e4cd879618a4/packages/jupyter-ai/jupyter_ai/models.py#L32-L38

Proposed Solution

Potential options:

  1. Option suggested by @3coins. Make AgentChatMessage.body a JSON object instead of a string. Expected format of the JSON data can be defined with Pydantic classes or JSON schemas (I prefer Pydantic classes).
   body: {
      "text": "Some text message",
      "error": {
         "type": "APIAuthenticationError",
         "message": "There was an issue with the API authentication."
      },
      "image": "image_url_if_applicable",
      ...
   }
  1. Add one additional field options that would be a JSON object and would contain all (expanding) additional options.
 class AgentChatMessage(BaseModel): 
     ...
     options: Dict[str, Any] 
  1. Add additional properties AgentChatMessage 1-by-1 as was attempted in this commit in #513 . Pros: straightforward approach. Cons: addition of further options would bloat the model
 class AgentChatMessage(BaseModel): 
     ...
     error_type: str
     another_option: ...
     ...

andrii-i avatar Dec 18 '23 18:12 andrii-i