LibreChat
LibreChat copied to clipboard
Published
20 hours ago •
danny-avila
danny-avila
[Question]: Can't use official retrieval plugin. Bug?
Contact Details
No response
What is your question?
After extensively reading the documentation for both the retrieval plugin and LibreChat, I successfully deployed the retrieval plugin API. I then used the Swagger web UI to insert and query information into the API's vector database, which worked as expected.
Following the LibreChat documentation, I added the plugin's manifest file to /api/app/clients/tools/.well-known and made sure that the filename matched the name_for_model property. The plugin now appears in the plugin store and can be installed without any problems. However, when I try to insert or query information, I consistently get Error: Request failed with status code 400.
Then, in an effort to resolve the issue, I updated the description_for_model property in the plugin manifest file to provide additional instructions for calling the API, but the error still persists.
While reviewing the retrieval API logs, I noticed that whenever the LLM attempts to use the retrieval plugin, it logs INFO: <my ip address>:55698 - "GET /openapi.yaml HTTP/1.1" 200 OK. I was expecting to see a "POST /query HTTP/1.1" 200 OK log entry, but this is not the case. It only logs the "GET /openapi.yaml" request.
For reference, I'm using gpt-3.5-turbo (with temperature 0) as the agent model and gpt-3.5-turbo-16k as the completion model.
I've taken care to follow all the necessary steps based on the documentation, but I must be missing or misunderstanding something. If there's any additional information or configuration I need to consider, I would appreciate any insights.
More Details
Librechat Logs
ask log
{
text: 'when is my birthday? use the retrieval plugin',
conversationId: null,
endpointOption: {
chatGptLabel: null,
promptPrefix: null,
tools: [ 'retrieval' ],
modelOptions: {
model: 'gpt-3.5-turbo-16k',
temperature: 0.8,
top_p: 1,
presence_penalty: 0,
frequency_penalty: 0
},
agentOptions: {
agent: 'functions',
skipCompletion: true,
model: 'gpt-3.5-turbo',
temperature: 0
}
}
}
Plugins sendMessage when is my birthday? use the retrieval plugin {
getIds: [Function: getIds],
user: '64d53ab7c22efbd7a631378b',
parentMessageId: '00000000-0000-0000-0000-000000000000',
conversationId: null,
overrideParentMessageId: null,
onAgentAction: [Function: onAgentAction],
onChainEnd: [Function: onChainEnd],
onStart: [Function: onStart],
chatGptLabel: null,
promptPrefix: null,
tools: [ 'retrieval' ],
modelOptions: {
model: 'gpt-3.5-turbo-16k',
temperature: 0.8,
top_p: 1,
presence_penalty: 0,
frequency_penalty: 0
},
agentOptions: {
agent: 'functions',
skipCompletion: true,
model: 'gpt-3.5-turbo',
temperature: 0
},
onProgress: [Function: wrapper],
abortController: AbortController { signal: AbortSignal { aborted: false } }
}
Loading history for conversation c9a73fb2-f91f-4b75-aa49-b3010080bb27 00000000-0000-0000-0000-000000000000
getTokenCountForMessage {
role: 'user',
content: 'when is my birthday? use the retrieval plugin'
}
freeAndResetAllEncoders: reached 25 encodings, resetting...
propertyTokenCounts [ 1, 9 ]
<---------------------------------DIFF--------------------------------->
Difference between payload (1) and orderedWithInstructions (1): 0
remainingContextTokens, this.maxContextTokens (1/2) 15985 15999
remainingContextTokens, this.maxContextTokens (2/2) 15985 15999
<-------------------------PAYLOAD/TOKEN COUNT MAP------------------------->
Payload: [
{
role: 'user',
content: 'when is my birthday? use the retrieval plugin',
tokenCount: 14
}
]
Token Count Map: { '7ef0fbe8-5426-4e13-93da-3e1bb5840d91': 14 }
Prompt Tokens 14 15985 15999
{ '7ef0fbe8-5426-4e13-93da-3e1bb5840d91': 14, instructions: undefined }
userMessage.tokenCount 14
[Meilisearch] Convo not found and will index c9a73fb2-f91f-4b75-aa49-b3010080bb27
createLLM: configOptions
{}
<-----Agent Model: gpt-3.5-turbo | Temp: 0 | Functions: true----->
files [
'Ai_PDF.json',
'Diagrams.json',
'Dr_Thoths_Tarot.json',
'DreamInterpreter.json',
'VoxScript.json',
'aitoolhunt.json',
'askyourpdf.json',
'drink_maestro.json',
'earthImagesAndVisualizations.json',
'image_prompt_enhancer.json',
'qrCodes.json',
'rephrase.json',
'retrieval.json',
'scholarai.json',
'uberchord.json',
'web_search.json'
]
{
Ai_PDF: [AsyncFunction (anonymous)],
Diagrams: [AsyncFunction (anonymous)],
Dr_Thoths_Tarot: [AsyncFunction (anonymous)],
DreamInterpreter: [AsyncFunction (anonymous)],
VoxScript: [AsyncFunction (anonymous)],
aitoolhunt: [AsyncFunction (anonymous)],
askyourpdf: [AsyncFunction (anonymous)],
drink_maestro: [AsyncFunction (anonymous)],
earthImagesAndVisualizations: [AsyncFunction (anonymous)],
image_prompt_enhancer: [AsyncFunction (anonymous)],
qrCodes: [AsyncFunction (anonymous)],
rephrase: [AsyncFunction (anonymous)],
retrieval: [AsyncFunction (anonymous)],
scholarai: [AsyncFunction (anonymous)],
uberchord: [AsyncFunction (anonymous)],
web_search: [AsyncFunction (anonymous)]
}
Requested Tools
[ 'retrieval' ]
Loaded Tools
[ 'retrieval' ]
Loaded agent.
Attempt 1 of 1
[chain/start] [1:chain:AgentExecutor] Entering Chain run with input: {
"input": "when is my birthday? use the retrieval plugin",
"signal": {},
"chat_history": []
}
[llm/start] [1:chain:AgentExecutor > 2:llm:ChatOpenAI] Entering LLM run with input: {
"messages": [
[
{
"lc": 1,
"type": "constructor",
"id": [
"langchain",
"schema",
"SystemMessage"
],
"kwargs": {
"content": "You are a helpful AI assistant.",
"additional_kwargs": {}
}
},
{
"lc": 1,
"type": "constructor",
"id": [
"langchain",
"schema",
"HumanMessage"
],
"kwargs": {
"content": "when is my birthday? use the retrieval plugin",
"additional_kwargs": {}
}
}
]
]
}
[llm/end] [1:chain:AgentExecutor > 2:llm:ChatOpenAI] [1.11s] Exiting LLM run with output: {
"generations": [
[
{
"text": "",
"message": {
"lc": 1,
"type": "constructor",
"id": [
"langchain",
"schema",
"AIMessage"
],
"kwargs": {
"content": "",
"additional_kwargs": {
"function_call": {
"name": "retrieval",
"arguments": "{\n \"query\": \"When is my birthday?\"\n}"
}
}
}
}
}
]
],
"llmOutput": {
"tokenUsage": {
"completionTokens": 19,
"promptTokens": 93,
"totalTokens": 112
}
}
}
Latest Agent Action {
tool: 'retrieval',
toolInput: { query: 'When is my birthday?' },
log: ''
}
[agent/action] [1:chain:AgentExecutor] Agent selected action: {
"tool": "retrieval",
"toolInput": {
"query": "When is my birthday?"
},
"log": ""
}
[tool/start] [1:chain:AgentExecutor > 3:tool:DynamicStructuredTool] Entering Tool run with input: "{"query":"When is my birthday?"}"
Attempting to load an OpenAPI 3.0.2 spec. This may result in degraded performance. Convert your OpenAPI spec to 3.1.0 for better support.
[chain/start] [1:chain:SequentialChain] Entering Chain run with input: {
"query": "when is my birthday? use the retrieval plugin\n\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\nPrioritize using responses for subsequent requests to better fulfill the query."
}
[chain/start] [1:chain:SequentialChain > 2:chain:LLMChain] Entering Chain run with input: {
"query": "when is my birthday? use the retrieval plugin\n\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\nPrioritize using responses for subsequent requests to better fulfill the query."
}
[llm/start] [1:chain:SequentialChain > 2:chain:LLMChain > 3:llm:ChatOpenAI] Entering LLM run with input: {
"messages": [
[
{
"lc": 1,
"type": "constructor",
"id": [
"langchain",
"schema",
"HumanMessage"
],
"kwargs": {
"content": "Use the provided API's to respond to this user query:\n\nwhen is my birthday? use the retrieval plugin\n\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\nPrioritize using responses for subsequent requests to better fulfill the query.",
"additional_kwargs": {}
}
}
]
]
}
[llm/error] [1:chain:SequentialChain > 2:chain:LLMChain > 3:llm:ChatOpenAI] [257ms] LLM run errored with error: "Request failed with status code 400"
[chain/error] [1:chain:SequentialChain > 2:chain:LLMChain] [260ms] Chain run errored with error: "Request failed with status code 400"
[chain/error] [1:chain:SequentialChain] [263ms] Chain run errored with error: "Request failed with status code 400"
[tool/error] [1:chain:AgentExecutor > 3:tool:DynamicStructuredTool] [422ms] Tool run errored with error: "Request failed with status code 400"
[chain/error] [1:chain:AgentExecutor] [1.55s] Chain run errored with error: "Request failed with status code 400"
Error: Request failed with status code 400
at createError (/app/node_modules/openai/node_modules/axios/lib/core/createError.js:16:15)
at settle (/app/node_modules/openai/node_modules/axios/lib/core/settle.js:17:12)
at IncomingMessage.handleStreamEnd (/app/node_modules/openai/node_modules/axios/lib/adapters/http.js:322:11)
at IncomingMessage.emit (node:events:525:35)
at endReadableNT (node:internal/streams/readable:1359:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
config: {
transitional: {
silentJSONParsing: true,
forcedJSONParsing: true,
clarifyTimeoutError: false
},
adapter: [Function: httpAdapter],
transformRequest: [ [Function: transformRequest] ],
transformResponse: [ [Function: transformResponse] ],
timeout: 0,
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
maxContentLength: -1,
maxBodyLength: -1,
validateStatus: [Function: validateStatus],
headers: {
Accept: 'application/json, text/plain, */*',
'Content-Type': 'application/json',
'User-Agent': 'OpenAI/NodeJS/3.3.0',
Authorization: 'Bearer sk-DFHu00A8H<redacted>R1GG2NI39',
'Content-Length': 1942
},
method: 'post',
data: '{"model":"gpt-3.5-turbo","temperature":0,"top_p":1,"frequency_penalty":0,"presence_penalty":0,"n":1,"stream":false,"functions":[{"name":"upsert_upsert_post","description":"Save chat information. Accepts an array of documents with text (potential questions + conversation text), metadata (source 'chat' and timestamp, no ID as this will be generated). Confirm with the user before saving, ask for more details/context.","parameters":{"type":"object","properties":{"data":{"type":"object","properties":{"documents":{"type":"array"}},"required":[],"additionalProperties":{}}},"required":["data"]}},{"name":"query_query_post","description":"Accepts search query objects array each with query and optional filter. Break down complex questions into sub-questions. Refine results by criteria, e.g. time / source, don't do this often. Split queries if ResponseTooLargeError occurs.","parameters":{"type":"object","properties":{"data":{"type":"object","properties":{"queries":{"type":"array"}},"required":[],"additionalProperties":{}}},"required":["data"]}}],"messages":[{"role":"user","content":"Use the provided API's to respond to this user query:\\n\\nwhen is my birthday? use the retrieval plugin\\n\\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\\nPrioritize using responses for subsequent requests to better fulfill the query."}]}',
url: 'https://api.openai.com/v1/chat/completions'
},
request: <ref *1> ClientRequest {
_events: [Object: null prototype] {
abort: [Function (anonymous)],
aborted: [Function (anonymous)],
connect: [Function (anonymous)],
error: [Function (anonymous)],
socket: [Function (anonymous)],
timeout: [Function (anonymous)],
finish: [Function: requestOnFinish]
},
_eventsCount: 7,
_maxListeners: undefined,
outputData: [],
outputSize: 0,
writable: true,
destroyed: true,
_last: false,
chunkedEncoding: false,
shouldKeepAlive: true,
maxRequestsOnConnectionReached: false,
_defaultKeepAlive: true,
useChunkedEncodingByDefault: true,
sendDate: false,
_removedConnection: false,
_removedContLen: false,
_removedTE: false,
strictContentLength: false,
_contentLength: 1942,
_hasBody: true,
_trailer: '',
finished: true,
_headerSent: true,
_closed: true,
socket: TLSSocket {
_tlsOptions: [Object],
_secureEstablished: true,
_securePending: false,
_newSessionPending: false,
_controlReleased: true,
secureConnecting: false,
_SNICallback: null,
servername: 'api.openai.com',
alpnProtocol: false,
authorized: true,
authorizationError: null,
encrypted: true,
_events: [Object: null prototype],
_eventsCount: 9,
connecting: false,
_hadError: false,
_parent: null,
_host: 'api.openai.com',
_closeAfterHandlingError: false,
_readableState: [ReadableState],
_maxListeners: undefined,
_writableState: [WritableState],
allowHalfOpen: false,
_sockname: null,
_pendingData: null,
_pendingEncoding: '',
server: undefined,
_server: null,
ssl: [TLSWrap],
_requestCert: true,
_rejectUnauthorized: true,
timeout: 5000,
parser: null,
_httpMessage: null,
[Symbol(res)]: [TLSWrap],
[Symbol(verified)]: true,
[Symbol(pendingSession)]: null,
[Symbol(async_id_symbol)]: -1,
[Symbol(kHandle)]: [TLSWrap],
[Symbol(lastWriteQueueSize)]: 0,
[Symbol(timeout)]: Timeout {
_idleTimeout: 5000,
_idlePrev: [TimersList],
_idleNext: [Timeout],
_idleStart: 2595312,
_onTimeout: [Function: bound ],
_timerArgs: undefined,
_repeat: null,
_destroyed: false,
[Symbol(refed)]: false,
[Symbol(kHasPrimitive)]: false,
[Symbol(asyncId)]: 15965,
[Symbol(triggerId)]: 15963
},
[Symbol(kBuffer)]: null,
[Symbol(kBufferCb)]: null,
[Symbol(kBufferGen)]: null,
[Symbol(kCapture)]: false,
[Symbol(kSetNoDelay)]: false,
[Symbol(kSetKeepAlive)]: true,
[Symbol(kSetKeepAliveInitialDelay)]: 1,
[Symbol(kBytesRead)]: 0,
[Symbol(kBytesWritten)]: 0,
[Symbol(connect-options)]: [Object]
},
_header: 'POST /v1/chat/completions HTTP/1.1\r\n' +
'Accept: application/json, text/plain, */*\r\n' +
'Content-Type: application/json\r\n' +
'User-Agent: OpenAI/NodeJS/3.3.0\r\n' +
'Authorization: Bearer sk-DFHu00A8H<redacted>R1GG2NI39\r\n' +
'Content-Length: 1942\r\n' +
'Host: api.openai.com\r\n' +
'Connection: keep-alive\r\n' +
'\r\n',
_keepAliveTimeout: 0,
_onPendingData: [Function: nop],
agent: Agent {
_events: [Object: null prototype],
_eventsCount: 2,
_maxListeners: undefined,
defaultPort: 443,
protocol: 'https:',
options: [Object: null prototype],
requests: [Object: null prototype] {},
sockets: [Object: null prototype] {},
freeSockets: [Object: null prototype],
keepAliveMsecs: 1000,
keepAlive: true,
maxSockets: Infinity,
maxFreeSockets: 256,
scheduling: 'lifo',
maxTotalSockets: Infinity,
totalSocketCount: 1,
maxCachedSessions: 100,
_sessionCache: [Object],
[Symbol(kCapture)]: false
},
socketPath: undefined,
method: 'POST',
maxHeaderSize: undefined,
insecureHTTPParser: undefined,
joinDuplicateHeaders: undefined,
path: '/v1/chat/completions',
_ended: true,
res: IncomingMessage {
_readableState: [ReadableState],
_events: [Object: null prototype],
_eventsCount: 4,
_maxListeners: undefined,
socket: null,
httpVersionMajor: 1,
httpVersionMinor: 1,
httpVersion: '1.1',
complete: true,
rawHeaders: [Array],
rawTrailers: [],
joinDuplicateHeaders: undefined,
aborted: false,
upgrade: false,
url: '',
method: null,
statusCode: 400,
statusMessage: 'Bad Request',
client: [TLSSocket],
_consuming: false,
_dumped: false,
req: [Circular *1],
responseUrl: 'https://api.openai.com/v1/chat/completions',
redirects: [],
[Symbol(kCapture)]: false,
[Symbol(kHeaders)]: [Object],
[Symbol(kHeadersCount)]: 40,
[Symbol(kTrailers)]: null,
[Symbol(kTrailersCount)]: 0
},
aborted: false,
timeoutCb: null,
upgradeOrConnect: false,
parser: null,
maxHeadersCount: null,
reusedSocket: true,
host: 'api.openai.com',
protocol: 'https:',
_redirectable: Writable {
_writableState: [WritableState],
_events: [Object: null prototype],
_eventsCount: 3,
_maxListeners: undefined,
_options: [Object],
_ended: true,
_ending: true,
_redirectCount: 0,
_redirects: [],
_requestBodyLength: 1942,
_requestBodyBuffers: [],
_onNativeResponse: [Function (anonymous)],
_currentRequest: [Circular *1],
_currentUrl: 'https://api.openai.com/v1/chat/completions',
[Symbol(kCapture)]: false
},
[Symbol(kCapture)]: false,
[Symbol(kBytesWritten)]: 0,
[Symbol(kNeedDrain)]: false,
[Symbol(corked)]: 0,
[Symbol(kOutHeaders)]: [Object: null prototype] {
accept: [Array],
'content-type': [Array],
'user-agent': [Array],
authorization: [Array],
'content-length': [Array],
host: [Array]
},
[Symbol(errored)]: null,
[Symbol(kHighWaterMark)]: 16384,
[Symbol(kUniqueHeaders)]: null
},
response: {
status: 400,
statusText: 'Bad Request',
headers: {
date: 'Tue, 15 Aug 2023 18:05:37 GMT',
'content-type': 'application/json',
'content-length': '265',
connection: 'keep-alive',
'access-control-allow-origin': '*',
'openai-organization': 'user-hkbdfgzgevahxkblkp8csj57',
'openai-processing-ms': '9',
'openai-version': '2020-10-01',
'strict-transport-security': 'max-age=15724800; includeSubDomains',
'x-ratelimit-limit-requests': '3500',
'x-ratelimit-limit-tokens': '90000',
'x-ratelimit-remaining-requests': '3499',
'x-ratelimit-remaining-tokens': '89770',
'x-ratelimit-reset-requests': '17ms',
'x-ratelimit-reset-tokens': '152ms',
'x-request-id': 'f59379fad3a69758c81896e15d145ab3',
'cf-cache-status': 'DYNAMIC',
server: 'cloudflare',
'cf-ray': '7f735de699efa6bb-GRU',
'alt-svc': 'h3=":443"; ma=86400'
},
config: {
transitional: [Object],
adapter: [Function: httpAdapter],
transformRequest: [Array],
transformResponse: [Array],
timeout: 0,
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
maxContentLength: -1,
maxBodyLength: -1,
validateStatus: [Function: validateStatus],
headers: [Object],
method: 'post',
data: '{"model":"gpt-3.5-turbo","temperature":0,"top_p":1,"frequency_penalty":0,"presence_penalty":0,"n":1,"stream":false,"functions":[{"name":"upsert_upsert_post","description":"Save chat information. Accepts an array of documents with text (potential questions + conversation text), metadata (source 'chat' and timestamp, no ID as this will be generated). Confirm with the user before saving, ask for more details/context.","parameters":{"type":"object","properties":{"data":{"type":"object","properties":{"documents":{"type":"array"}},"required":[],"additionalProperties":{}}},"required":["data"]}},{"name":"query_query_post","description":"Accepts search query objects array each with query and optional filter. Break down complex questions into sub-questions. Refine results by criteria, e.g. time / source, don't do this often. Split queries if ResponseTooLargeError occurs.","parameters":{"type":"object","properties":{"data":{"type":"object","properties":{"queries":{"type":"array"}},"required":[],"additionalProperties":{}}},"required":["data"]}}],"messages":[{"role":"user","content":"Use the provided API's to respond to this user query:\\n\\nwhen is my birthday? use the retrieval plugin\\n\\n||>Instructions: Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.\\nPrioritize using responses for subsequent requests to better fulfill the query."}]}',
url: 'https://api.openai.com/v1/chat/completions'
},
request: <ref *1> ClientRequest {
_events: [Object: null prototype],
_eventsCount: 7,
_maxListeners: undefined,
outputData: [],
outputSize: 0,
writable: true,
destroyed: true,
_last: false,
chunkedEncoding: false,
shouldKeepAlive: true,
maxRequestsOnConnectionReached: false,
_defaultKeepAlive: true,
useChunkedEncodingByDefault: true,
sendDate: false,
_removedConnection: false,
_removedContLen: false,
_removedTE: false,
strictContentLength: false,
_contentLength: 1942,
_hasBody: true,
_trailer: '',
finished: true,
_headerSent: true,
_closed: true,
socket: [TLSSocket],
_header: 'POST /v1/chat/completions HTTP/1.1\r\n' +
'Accept: application/json, text/plain, */*\r\n' +
'Content-Type: application/json\r\n' +
'User-Agent: OpenAI/NodeJS/3.3.0\r\n' +
'Authorization: Bearer sk-DFHu00A8H<redacted>R1GG2NI39\r\n' +
'Content-Length: 1942\r\n' +
'Host: api.openai.com\r\n' +
'Connection: keep-alive\r\n' +
'\r\n',
_keepAliveTimeout: 0,
_onPendingData: [Function: nop],
agent: [Agent],
socketPath: undefined,
method: 'POST',
maxHeaderSize: undefined,
insecureHTTPParser: undefined,
joinDuplicateHeaders: undefined,
path: '/v1/chat/completions',
_ended: true,
res: [IncomingMessage],
aborted: false,
timeoutCb: null,
upgradeOrConnect: false,
parser: null,
maxHeadersCount: null,
reusedSocket: true,
host: 'api.openai.com',
protocol: 'https:',
_redirectable: [Writable],
[Symbol(kCapture)]: false,
[Symbol(kBytesWritten)]: 0,
[Symbol(kNeedDrain)]: false,
[Symbol(corked)]: 0,
[Symbol(kOutHeaders)]: [Object: null prototype],
[Symbol(errored)]: null,
[Symbol(kHighWaterMark)]: 16384,
[Symbol(kUniqueHeaders)]: null
},
data: { error: [Object] }
},
isAxiosError: true,
toJSON: [Function: toJSON],
attemptNumber: 1,
retriesLeft: 6
}
Failed to parse JSON: SyntaxError: Unexpected token 'w', "when is my"... is not valid JSON
at JSON.parse (<anonymous>)
at findMessageContent (/app/api/utils/findMessageContent.js:22:30)
at PluginsClient.executorCall (/app/api/app/clients/PluginsClient.js:297:25)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async PluginsClient.sendMessage (/app/api/app/clients/PluginsClient.js:414:5)
at async ask (/app/api/server/routes/ask/gptPlugins.js:222:20)
at async /app/api/server/routes/ask/gptPlugins.js:67:10
Stream ended
getTokenCountForMessage {
role: 'assistant',
content: 'Encountered an error while attempting to respond. Error: Request failed with status code 400'
}
propertyTokenCounts [ 1, 19 ]
[Meilisearch] Convo not found and will index c9a73fb2-f91f-4b75-aa49-b3010080bb27
CLIENT RESPONSE
{
messageId: 'c8fc40c2-f228-440a-ae30-c845d1b8c8d6',
conversationId: 'c9a73fb2-f91f-4b75-aa49-b3010080bb27',
parentMessageId: '7ef0fbe8-5426-4e13-93da-3e1bb5840d91',
isCreatedByUser: false,
model: 'gpt-3.5-turbo-16k',
sender: 'Assistant',
promptTokens: 14,
text: 'Encountered an error while attempting to respond. Error: Request failed with status code 400',
completionTokens: 24,
output: 'Encountered an error while attempting to respond. Error: Request failed with status code 400',
intermediateSteps: [
{
tool: 'retrieval',
toolInput: { query: 'When is my birthday?' },
log: ''
}
],
errorMessage: 'Request failed with status code 400'
}
CONVERSATION TITLE Error Encountered While Retrieving Birthday
[Meilisearch] Convo not found and will index c9a73fb2-f91f-4b75-aa49-b3010080bb27
What is the main subject of your question?
Other
Screenshots
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
It's fetching the openapi spec to generate the functions it will use.
"GET /openapi.yaml HTTP/1.1" 200 OK
It would be helpful to see the logs from the LibreChat side when it fails, but I suspect it's having trouble either interpreting the spec or there is a server url mismatch.
do you also mind sharing what your .well-known/openapi.yaml looks like?
See this part for more detail: https://docs.librechat.ai/features/plugins/chatgpt_plugins_openapi.html#custom-openapi-spec-files
Sure! Here:
openapi.yaml
openapi: 3.0.2
info:
title: Retrieval Plugin API
description: A retrieval API for querying and filtering documents based on natural language queries and metadata
version: 1.0.0
servers:
- url: https://retrieval-plugin.mydomain.tld
paths:
/upsert:
post:
summary: Upsert
description: Save chat information. Accepts an array of documents with text (potential questions + conversation text), metadata (source 'chat' and timestamp, no ID as this will be generated). Confirm with the user before saving, ask for more details/context.
operationId: upsert_upsert_post
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/UpsertRequest"
required: true
responses:
"200":
description: Successful Response
content:
application/json:
schema:
$ref: "#/components/schemas/UpsertResponse"
"422":
description: Validation Error
content:
application/json:
schema:
$ref: "#/components/schemas/HTTPValidationError"
security:
- HTTPBearer: []
/query:
post:
summary: Query
description: Accepts search query objects array each with query and optional filter. Break down complex questions into sub-questions. Refine results by criteria, e.g. time / source, don't do this often. Split queries if ResponseTooLargeError occurs.
operationId: query_query_post
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/QueryRequest"
required: true
responses:
"200":
description: Successful Response
content:
application/json:
schema:
$ref: "#/components/schemas/QueryResponse"
"422":
description: Validation Error
content:
application/json:
schema:
$ref: "#/components/schemas/HTTPValidationError"
security:
- HTTPBearer: []
components:
schemas:
Document:
title: Document
required:
- text
type: object
properties:
id:
title: Id
type: string
text:
title: Text
type: string
metadata:
$ref: "#/components/schemas/DocumentMetadata"
DocumentChunkMetadata:
title: DocumentChunkMetadata
type: object
properties:
source:
$ref: "#/components/schemas/Source"
source_id:
title: Source Id
type: string
url:
title: Url
type: string
created_at:
title: Created At
type: string
author:
title: Author
type: string
document_id:
title: Document Id
type: string
DocumentChunkWithScore:
title: DocumentChunkWithScore
required:
- text
- metadata
- score
type: object
properties:
id:
title: Id
type: string
text:
title: Text
type: string
metadata:
$ref: "#/components/schemas/DocumentChunkMetadata"
embedding:
title: Embedding
type: array
items:
type: number
score:
title: Score
type: number
DocumentMetadata:
title: DocumentMetadata
type: object
properties:
source:
$ref: "#/components/schemas/Source"
source_id:
title: Source Id
type: string
url:
title: Url
type: string
created_at:
title: Created At
type: string
author:
title: Author
type: string
DocumentMetadataFilter:
title: DocumentMetadataFilter
type: object
properties:
document_id:
title: Document Id
type: string
source:
$ref: "#/components/schemas/Source"
source_id:
title: Source Id
type: string
author:
title: Author
type: string
start_date:
title: Start Date
type: string
end_date:
title: End Date
type: string
HTTPValidationError:
title: HTTPValidationError
type: object
properties:
detail:
title: Detail
type: array
items:
$ref: "#/components/schemas/ValidationError"
Query:
title: Query
required:
- query
type: object
properties:
query:
title: Query
type: string
filter:
$ref: "#/components/schemas/DocumentMetadataFilter"
top_k:
title: Top K
type: integer
default: 3
QueryRequest:
title: QueryRequest
required:
- queries
type: object
properties:
queries:
title: Queries
type: array
items:
$ref: "#/components/schemas/Query"
QueryResponse:
title: QueryResponse
required:
- results
type: object
properties:
results:
title: Results
type: array
items:
$ref: "#/components/schemas/QueryResult"
QueryResult:
title: QueryResult
required:
- query
- results
type: object
properties:
query:
title: Query
type: string
results:
title: Results
type: array
items:
$ref: "#/components/schemas/DocumentChunkWithScore"
Source:
title: Source
enum:
- email
- file
- chat
type: string
description: An enumeration.
UpsertRequest:
title: UpsertRequest
required:
- documents
type: object
properties:
documents:
title: Documents
type: array
items:
$ref: "#/components/schemas/Document"
UpsertResponse:
title: UpsertResponse
required:
- ids
type: object
properties:
ids:
title: Ids
type: array
items:
type: string
ValidationError:
title: ValidationError
required:
- loc
- msg
- type
type: object
properties:
loc:
title: Location
type: array
items:
anyOf:
- type: string
- type: integer
msg:
title: Message
type: string
type:
title: Error Type
type: string
securitySchemes:
HTTPBearer:
type: http
scheme: bearer
Also the manifest file in case you need it:
ai-plugin.json (/api/app/clients/tools/.well-known/retrieval.json)
{
"schema_version": "v1",
"name_for_model": "retrieval",
"name_for_human": "Retrieval Plugin",
"description_for_model": "Plugin for searching through the user's documents (such as files, emails, and more) to find answers to questions and retrieve relevant information. Use it whenever a user asks something that might be found in their personal information, or asks you to save information for later. The '/upsert' path with a POST request saves chat information by accepting an array of documents with text and metadata. The '/query' path with a POST request accepts search query objects array each with a query and optional filter, and returns documents based on the queries. Ask the user for more details and/or context. Always confirm with the user before saving.",
"description_for_human": "Search through your documents.",
"auth": {
"type": "user_http",
"authorization_type": "bearer"
},
"api": {
"type": "openapi",
"url": "https://retrieval-plugin.mydomain.tld/.well-known/openapi.yaml",
"has_user_authentication": false
},
"logo_url": "https://retrieval-plugin.mydomain.tld/.well-known/logo.png",
"contact_email": "[email protected]",
"legal_info_url": "[email protected]"
}
Note: Figured I'd change the actual API URL from the spec/manifest file before posting here for security reasons, even though it's for testing purposes only.
"auth": { "type": "user_http", "authorization_type": "bearer" },
I can see there potentially being an issue with authorization and no verification token
I noticed the retrieval plugin requires a BEARER_TOKEN, it shows in retrieval docs it's required.
I'm only handling the specific setup shown here atm, because it's how a lot of chatgpt plugins authenticate. You can mirror the setup from the librechat docs.
it would also help to see the logs from librechat when you experience this:
my bad, i didn't see you already posted the logs. will look at them now
Ok so from what I can tell OpenAI is rejecting the function call even tho it might be correct because the description may be too long, in particular for query_query_post.
you can try including the openapi schema manually and try making that shorter. OpenAI has no documentation on this but I've discovered in the past a hard limit on the function description and it's pretty short.
Part of this will need reworking on my end as I will have to mess with langchain/the openAI api in a more granular fashion and I hope to get to it soon. it should make the chatgpt plugins more reliable when I get to it
Apologies for not highlighting the logs more clearly in the issue description, my bad. I'll make sure they're more visible next time.
Regarding the BEARER_TOKEN, I followed the librechat docs and tried that approach initially. It didn't seem to work or change anything, so I reversed it. I'll revisit this and try again, ensuring my setup matches the documentation.
As for manually including the OpenAPI schema, if by that you mean adding a /api/app/clients/tools/.well-known/openapi/retrieval.yaml file with the spec, and adjusting the manifest as follows:
"api": {
"type": "openapi",
"url": "retrieval.yaml",
"has_user_authentication": false
},
I did attempt that, but it didn't work either, unfortunately. However, I'll try shortening the query_query_post and let you know if that helps.
Made the following changes:
- Implemented the BEARER_TOKEN as outlined in the LibreChat docs.
- Simplified the descriptions for the /upsert and /query endpoints in the OpenAPI spec, and renamed
query_query_posttoquery_post, likewise for upsert. - Added the spec to
/api/app/clients/tools/.well-known/openapi/retrieval.yamland modified the manifest accordingly.
However, still not working. Resulting in the same error: Request failed with status code 400.
Here's the updated spec and manifest:
Updated OpenAPI spec (abbreviated)
paths:
/upsert:
post:
summary: Upsert
# Shortened the description
description: Save chat documents with text, metadata like source and timestamp. Confirm with user before saving.
operationId: upsert_post # previously was 'upsert_upsert_post'
# Omitted for brevity
/query:
post:
summary: Query
# Shortened the description
description: Accepts search queries with text and filters. Break into sub-queries if needed. Refine by criteria like time and source. Handle large responses.
operationId: query_post # previously was 'query_query_post'
# Omitted for brevity
Updated manifest file (abbreviated)
{
// Ommited for brevity
"auth": {
"type": "service_http",
"authorization_type": "bearer",
"verification_tokens": {
"openai": "eyJhb<token>DQI"
}
},
"api": {
"type": "openapi",
"url": "retrieval.yaml",
"has_user_authentication": false
}
// Omitted for brevity
}
I've been looking into the LibreChat logs, and it seems that createOpenAPIChain might be used internally for API calls. I'm not familiar with the codebase, and I know there's a lot more prompt injection, it's not that simple, so I apologize if I'm mistaken. I decided to test it out in a repl:
import { createOpenAPIChain } from "langchain/chains";
import { ChatOpenAI } from "langchain/chat_models/openai";
const apiKey = "sk-..."
const apiSpec = "https://retrieval-plugin.mydomain.tld/.well-known/openapi.yaml";
const chatModel = new ChatOpenAI({ modelName: "gpt-3.5-turbo-0613", temperature: 0, openAIApiKey: apiKey });
const chain = await createOpenAPIChain(apiSpec, {
verbose: true,
llm: chatModel,
headers: {
authorization: "Bearer eyJhb<token>DQI",
}
});
try {
const result = await chain.run(`Save this for later: my birthday is on august 4th, 2001`);
console.log(JSON.stringify(result, null, 2));
} catch (e) {
console.dir(e.response.data);
}
Got this error:
{
error: {
message: [
"Invalid schema for function 'upsert_post': In context=('properties', 'data', 'properties', 'documents'), array schema missing items"
],
type: 'invalid_request_error',
param: null,
code: null
}
}
I'm not sure if this is relevant at all, but it may be why we're seeing the bad request error? I've been googling for a while, couldn't find anything.
I've been looking into the LibreChat logs, and it seems that
createOpenAPIChainmight be used internally for API calls. I'm not familiar with the codebase, and I know there's a lot more prompt injection, it's not that simple, so I apologize if I'm mistaken. I decided to test it out in a repl:import { createOpenAPIChain } from "langchain/chains"; import { ChatOpenAI } from "langchain/chat_models/openai"; const apiKey = "sk-..." const apiSpec = "https://retrieval-plugin.mydomain.tld/.well-known/openapi.yaml"; const chatModel = new ChatOpenAI({ modelName: "gpt-3.5-turbo-0613", temperature: 0, openAIApiKey: apiKey }); const chain = await createOpenAPIChain(apiSpec, { verbose: true, llm: chatModel, headers: { authorization: "Bearer eyJhb<token>DQI", } }); try { const result = await chain.run(`Save this for later: my birthday is on august 4th, 2001`); console.log(JSON.stringify(result, null, 2)); } catch (e) { console.dir(e.response.data); }Got this error:
{ error: { message: [ "Invalid schema for function 'upsert_post': In context=('properties', 'data', 'properties', 'documents'), array schema missing items" ], type: 'invalid_request_error', param: null, code: null } }I'm not sure if this is relevant at all, but it may be why we're seeing the bad request error? I've been googling for a while, couldn't find anything.
in your example, the LLM is generating a payload that doesn't conform to the schema spec, and langchain is strict in aligning the two, so it will throw a schema error (createOpenAPIChain).
In the original error, (status code 400), that is a problem with OpenAI API and the "function" payload. The plugin is converted to a function as understood by the OpenAI API. Note how the API reference says nothing about function description length, but there is a hard limit I've encountered on top of other issues with function formation that are not documented by OpenAI.
Since this is an "official" OpenAI chatgpt plugin, I think it will be worth it to convert this to a langchain tool for reliability
Also other than this, my work with the chatgpt plugins is a first iteration. I've been making notes on how to improve it, including with issues like this where the problem originates with a bad API call to OpenAI for generation, and also focusing on better prompt injecting as you mentioned. I'm wrapping up other features, but will tackle this soon since I already started on it, too.
@veigamann I made a lot of general improvements to the handling of plugins in https://github.com/danny-avila/LibreChat/pull/845, which is now merged
Let me know how the retrieval plugin fares as I haven't had time to set it up.
Note that file support/vectorization is a next major step I'm taking with the project.
Sorry for the delay, I will try it out later today or tomorrow in the afternoon/evening and get back to you.
Sorry for the delay, I will try it out later today or tomorrow in the afternoon/evening and get back to you.
No worries, thanks for doing that!