documate icon indicating copy to clipboard operation
documate copied to clipboard

Published 20 hours ago • verified publisher icon aircodelabs

BadRequestError

Open snowinszu opened this issue 2 years ago • 2 comments
trafficstars

What operating system are you using?

Mac

Describe the bug

BadRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 4137 tokens. Please reduce the length of the messages.

Steps to reproduce

normal usage

Reproduction Link

No response

snowinszu avatar Nov 14 '23 08:11 snowinszu

Approximately how large is your documentation website? And how long was the question?

Maybe the embeddings query returned a full 4097 tokens and then the question was put on top.

Maybe reducing the MAX_TOKEN_PER_CHUNK from 8191 to something smaller will help in upload.js https://github.com/AirCodeLabs/documate/blob/b3a0c5210426f06cffef20380952f5c739b9c77b/backend/upload.js#L7C7-L7C26

Edit: On second thought, the ask.js function is supposed to limit the data from the embeddings at 1500 tokens 🤔 So maybe there is an issue somewhere else. Ensure you have the latest version with the token limit

mathiash98 avatar Nov 29 '23 15:11 mathiash98

Thanks, I will take a trial.

snowinszu avatar Nov 29 '23 17:11 snowinszu