pipelines
pipelines copied to clipboard
open-webui
Inlet and outlet not called using a pipeline as a base_model to create custom model
Hello! When I create a model by going to Workspaces -> Model -> Create a Model and selecting the model ID of a custom pipeline as the base model, the inlet and outlet functions don't seem to be triggered.
Any insights on why this might be happening? Thank you!
Due to company policies, I had to remove some code related to system prompt construction.
Currently, the main issue is that when using the pipeline directly, the self.file_contents variable updates correctly. However, if I wrap the pipeline to create a custom model from the UI, the inlet (and the oultet) is not called, and the self.file_contents list doesn’t update across different chats.
Here a code example
"""
title: pipeline
author: user
version: 1.0
license: MIT
description: A pipeline
requirements: langfuse, openai
"""
from typing import List, Optional, Union, Generator, Iterator
import os
import uuid
from pydantic import BaseModel
import openai
import re
from langfuse import Langfuse
from langfuse.api.resources.commons.errors.unauthorized_error import UnauthorizedError
import os
from utils.pipelines.main import get_last_assistant_message
# print Setup
...
# OIFile Class to manage openwebui file data
class OIFile:
def __init__(self, file_id: str, filename: str, content: str):
self.id = file_id
self.filename = filename
self.content = content
def __repr__(self):
return f"File(id={self.id}, filename={self.filename}, content={len(self.content)} bytes)"
class Pipeline:
class Valves(BaseModel):
LLM_BASE_URL: str
LLM_API_KEY: str
LLM_MODEL_NAME: str
LLM_TEMPERATURE: float
LLM_MAX_TOKENS: int
def __init__(self):
self.name = "pipeline"
self.valves = self._initialize_valves()
self.file_contents = [OIFile]
def _initialize_valves(self) -> Valves:
return self.Valves(
LLM_BASE_URL=os.getenv("LLM_BASE_URL", "url"),
LLM_API_KEY=os.getenv("LLM_API_KEY", "empty"),
LLM_MODEL_NAME=os.getenv(
"LLM_MODEL_NAME", "meta-llama/Llama-3.2-3B-Instruct"
),
LLM_TEMPERATURE=float(os.getenv("LLM_TEMPERATURE", 0)),
LLM_MAX_TOKENS=int(os.getenv("LLM_MAX_TOKENS", 10000)),
)
async def on_startup(self):
print.info(f"Server {self.name} is starting.")
async def on_shutdown(self):
print.info(f"Server {self.name} is shutting down.")
async def on_valves_updated(self):
print.info("Valves updated.")
async def inlet(self, body: dict, user: dict) -> dict:
print.info("Processing inlet request")
# Extract file info for all files in the body
self.file_contents = self._extract_file_info(body)
# Log the extracted file information
for file in self.file_contents:
print.info(
f"File info extracted: ID={file.id}, Filename={file.filename}, Content size={len(file.content)} bytes"
)
return body
def _extract_file_info(self, body: dict) -> list:
"""Extracts the file info from the request body for all files."""
files = []
for file_data in body.get("files", []):
file = file_data["file"]
file_id = file["id"]
filename = file["filename"]
file_content = file["data"]["content"]
# Create a OIFile object and append it to the list
files.append(OIFile(file_id, filename, file_content))
return files
def pipe(
self, body: dict, user_message: str, model_id: str, messages: List[dict]
) -> Union[str, Generator, Iterator]:
print.info("Starting PIPE process")
# Extract parameters from body with default fallbacks
stream = body.get("stream", True)
max_tokens = body.get("max_tokens", self.valves.LLM_MAX_TOKENS)
temperature = body.get("temperature", self.valves.LLM_TEMPERATURE)
system_prompt = self._extract_system_prompt(messages)
# Generate and update the system prompt if required
if self.file_contents:
# build or retrieve a prompt
# adding file contents
# Call the LLM API
return self._call_openai_api(messages, max_tokens, temperature, stream)
def _call_openai_api(
self,
messages: List[dict],
max_tokens: int,
temperature: float,
stream: bool,
) -> Union[str, Generator, Iterator]:
client = openai.Client(
api_key=self.valves.LLM_API_KEY,
base_url=self.valves.LLM_BASE_URL,
)
try:
# Call OpenAI API with the prepared parameters
response = client.chat.completions.create(
model=self.valves.LLM_MODEL_NAME,
messages=messages,
max_tokens=max_tokens,
temperature=temperature,
stream=stream,
stream_options={"include_usage": True},
)
return response
except Exception as e:
print.error(f"Error during OpenAI API call: {e}")
return f"Error: {e}"
async def outlet(self, body: dict, user: Optional[dict] = None) -> dict:
print(f"outlet:{__name__}")
print(f"Received body: {body}")
self.file_contents=[]
return body
Sorry, I use wrap but I should use another term. When I create a model by going to Workspaces -> Model -> Create a Model and selecting the model ID of a custom pipeline as the base model, the inlet and outlet functions don't seem to be triggered. The pipeline used is the one showed above (plus some function to extract the system prompt and adding the file contents and other details). When I create the model as described at the beginning of this comment, the function inlet and outlet are not triggered
Moreover, i have seen this behaviour: In this custom pipeline setup, the workflow is as follows:
Documents are loaded and fed into OpenWebUI.
OpenWebUI encodes the documents, splits them into chunks, and stores them.
The /inlet endpoint on the pipeline is called.
OpenWebUI then performs the RAG operation.
The pipeline’s /pipe endpoint is triggered, displaying the response on OpenWebUI.
Finally, the client calls the /outlet endpoint to retrieve the result.
Question on Implementation:
Why do the /inlet, /pipe, and /outlet endpoints each have different request bodies?
This is a major issue since without the inlet there's no chat_id. With no chat_id, there's no way to keep track of the conversation when what you send to the user and what you send to the LLM are different, which in my case, all the time, with tool outputs being truncated.
