aws-samples
Add Agents to Model List
Issue #, if available: Closes #165.
Description of changes:
Changes were largely lifted from the PR still open by @dhapola. Per our issue, we just want to be able to chat with agents, so I lifted their bedrock_agent.py with the knowledge base parts removed from there and settings.py. I also carried his ModelManager into the type(s) in bedrock.py.
I haven't worked much with async before, but I aligned it with the functions and worked on their innards until the interpreter didn't yell at me. At least in my testing, whether it's a foundation model or a Bedrock agent, the logs and I/O in Open-WebUI look correct to me. Did I miss anything?
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Hi @herter4171-kp ! Any chance of getting this merged? I've tried your PR in action together with open-webui. It displays the agents in the model list, but when I try to interact with an agent, I get the error:
Response payload is not completed: <TransferEncodingError: 400, message='Not enough data to satisfy transfer length header.'>
@weekens, the last I saw, someone said it wasn't going to be merged due to not fitting the demo, but I fail to see their comment as of now. The error you're getting is because we add the user's name to the schema passed to the bedrock API to be used as the session ID. To enable that behavior, use the following function on agents in Open-WebUI. The header stuff is useless and could be taken out.
from flask import request
import logging
from typing import Optional
from pydantic import BaseModel
import json
class Filter:
# Valves: Configuration options for the filter
class Valves(BaseModel):
pass
def __init__(self):
# Initialize valves (optional configuration for the Filter)
self.valves = self.Valves()
def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
# This is where you manipulate user inputs.
if __user__:
username = __user__["name"]
else:
username = request.headers.get("X-User-Name")
body["session_id"] = username if username else "default"
print(json.dumps(body, indent=2))
return body