[Howto]: reduce token

[Morphmor]

Version

VisualStudio Code extension

Operating System

MacOS

Your question

It always get stuck and give me that error. Is there a way to increase the tokens taht can be used for this step ? Is Gpt-pilot always sending the full code including all files with every step end every request? Can i somehow tell Gpt-pilot to only send the needed information for the present task?

Error calling LLM API: The request exceeded the maximum token limit (request size: 8237) tokens.

--------- LLM Reached Token Limit ---------- Can I retry implementing the entire development step?

[neffetzz]

Hi, I think, I had the same problem like @Morphmor.

I'm using Anthropic as LLM-Server. It works fine untill I get the Error 429. Error 429 means that:

"...Number of request tokens has exceeded your rate limit (https://docs.anthropic.com/claude/reference/rate-limits). Please reduce the the prompt length or the maximum tokens requested, or try again later..."

I set MAX_TOKENS=4096.

Any ideas or suggestions?

[Wladastic]

You may have to rewrite a couple of prompts for this. Try increasing the context length in your llm to at least 17k. The code for doing requests is faulty as well, there is no truncation length given to the llm api, which means that if your AI has the token limit of 8k and receives 7k, it will still try to produce more. You can reduce this behaviour by setting the max_new_tokens to 1024 or 2048 for example.

Do not reduce MAX_TOKENS to 4096 as this makes gpt-pilots prompts nearly unusable. If you give your ai 1500 tokens as system prompt, then 8000 tokens context, then it is supposed to respond in 2048 tokens and 600 tokens at least, you will already reach the limit every time.

[invisiblepancake]

it was having python enviorment. made sude to root p to the project root directory. not one slip out of there? noh worries looking thru manuals to an pearl command sorted sciprts etc =)

poetry scripts/setup | from ur gpt`d dir

[neffetzz]

Hi @Wladastic , thank you for your comment. I set the Parameter „max_new_token=1024“ and gpt-pilot was able to create the First app.👌

[invisiblepancake]

if ur using apple hardware.... ive having issues with preinstalled plingdows to get elevated pric for scrape install etc

[invisiblepancake]

try getting an workstation with none preinstalled like anything rly. and build an linux os env =)

[neffetzz]

Hi,

Pythagora works best on my PC with the Anthropic API when I'm using the following .env settings:

/###########################################################################
## ANTHROPIC ##
ANTHROPIC_API_KEY=sk-ant-1111-11111111111111111111111111111111111111
ANTHROPIC_ENDPOINT=
MODEL_NAME=anthropic/claude-3-haiku-20240307
MAX_TOKENS=4096
MAX_NEW_TOKENS=1024 
###########################################################################