gpt-pilot
gpt-pilot copied to clipboard
Pythagora-io
[Bug]: Token Error Limit
Version
VisualStudio Code extension
Operating System
MacOS
What happened?
This has been a problem I've always seemed to have with GPT-Pilot, and I hope there's a workaround?
I always seem to end up hitting a token limit. It's always at the same kind of point in the dev cycle (each project is a different point, but it always seems to break at around the same point).
In this instance:
"gpt-pilot/pilot/utils/llm_connection.py", line 219, in wrapper raise TokenLimitError(get_tokens_in_messages_from_openai_error(err_str), MAX_GPT_MODEL_TOKENS)
helpers.exceptions.TokenLimitError.TokenLimitError: Token limit error happened with 264655/8192 tokens in messages!
Yeah, that's a lot of tokens over the limit! I'm not doing anything weird or wonderful, just a react webapp. In this most recent example it's before what would be getting the sockets set up for i/o.
Is there anything that can be done? I noticed that Tiktoken is on the installed list, but it doesn't seem to be able to catch this before it makes the attempt and exits-with-error.
TIA!
This is likely because we're sending the full directory strucutre/contents to the LLM so it knows where to make the changes.
We're working on selecting only appropriate files (for the task at hand) for the context, which should lower the token count, decrease the cost, and make GPT Pilot faster.
Hi @senko,
Thanks for your work. Do you see any ETA on this? I am constantly bumping into the token limit error before even starting any development in react native. Selecting the important files to send would be a huge improvement. Currently I can not even move to the first dev steps after setting up the react env and installations.
Cheers
@Zol1Magyar this might be helpful: https://www.youtube.com/watch?v=GMch80e69nQ