AutoGPT
AutoGPT copied to clipboard
Significant-Gravitas
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens
⚠️ Search for existing issues first ⚠️
- [X] I have searched the existing issues, and there is no existing issue for my problem
Which Operating System are you using?
Windows
Which version of Auto-GPT are you using?
Latest Release
GPT-3.5
Steps to reproduce 🕹
Building a business plan give me this error: openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 10134 tokens. Please reduce the length of the messages.
Current behavior 😯
CRITICISM: We need to ensure that our Python script is optimized for performance and that it is based on accurate and reliable information. We also need to be mindful of potential legal and regulatory challenges, and ensure that our technology infrastructure is secure and reliable.
NEXT ACTION: COMMAND = write_to_file ARGUMENTS = {'filename': 'C:\Auto-GPT-0.3.0\autogpt\auto_gpt_workspace\newproject.py', 'text': 'def generate_improved_code(suggestions, code):\n # TODO: Add code here\n\n return improved_code\n'}
SYSTEM: Command write_to_file returned: File written to successfully.
Traceback (most recent call last):
File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Auto-GPT-0.3.0\autogpt_main.py", line 5, in
Expected behavior 🤔
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 10134 tokens. Please reduce the length of the messages.
I have followed this instructions and still havent worked: https://github.com/Significant-Gravitas/Auto-GPT/discussions/1061
Your prompt 📝
# Paste your prompt here
CRITICISM: We need to ensure that our Python script is optimized for performance and that it is based on accurate and reliable information. We also need to be mindful of potential legal and regulatory challenges, and ensure that our technology infrastructure is secure and reliable.
NEXT ACTION: COMMAND = write_to_file ARGUMENTS = {'filename': 'C:\Auto-GPT-0.3.0\autogpt\auto_gpt_workspace\newproject.py', 'text': 'def generate_improved_code(suggestions, code):\n # TODO: Add code here\n\n return improved_code\n'}
SYSTEM: Command write_to_file returned: File written to successfully.
Traceback (most recent call last):
File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Auto-GPT-0.3.0\autogpt_main.py", line 5, in
Your Logs 📒
<insert your logs here>
CRITICISM: We need to ensure that our Python script is optimized for performance and that it is based on accurate and reliable information. We also need to be mindful of potential legal and regulatory challenges, and ensure that our technology infrastructure is secure and reliable.
NEXT ACTION: COMMAND = write_to_file ARGUMENTS = {'filename': 'C:\Auto-GPT-0.3.0\autogpt\auto_gpt_workspace\newproject.py', 'text': 'def generate_improved_code(suggestions, code):\n # TODO: Add code here\n\n return improved_code\n'}
SYSTEM: Command write_to_file returned: File written to successfully.
Traceback (most recent call last):
File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Auto-GPT-0.3.0\autogpt_main.py", line 5, in
Not sure if there is a solution or workaround to this....my program exits whenever it hits this limit midway...
Same problem here, tried a lot o f things but no solution at the moment. I tried to ask autogpt to cut all files in max 4000 tokens before reading as it was told in a comment, but it did not work.
the issue is in summary_memory.py -> update_running_summary:
when adding "Latest Development:" to the prompt for the history, after running auto-gpt for a while there are more and more entires... need to limit the history which is kept, otherwise prompt gets too long
(that's at least the issue each time i run into this error)
anyone wants to have a go at a patch? (havent contributed any fixes yet, but done a few locally for testing, could have a go at it)
the issue is in summary_memory.py -> update_running_summary:
when adding "Latest Development:" to the prompt for the history, after running auto-gpt for a while there are more and more entires... need to limit the history which is kept, otherwise prompt gets too long
(that's at least the issue each time i run into this error)
anyone wants to have a go at a patch? (havent contributed any fixes yet, but done a few locally for testing, could have a go at it)
+1, has intesest to try, thanks
Who has a summary of the game plan for handling aip_requestor.py and memory token overruns? I have not had any long term (more than 10 minutes) run go by without exhausting token limit and crashing instead of trying to automatically handle the exception or error and retry by auto condensing the prompts generated then trying again or opened a new agent to run part of the prompt and another for the rest of the prompt in chunks of XXX... anyhow - there has got to be some workarounds on this that are not too difficult to implement.
Not sure if there is a solution or workaround to this....my program exits whenever it hits this limit midway...
any luck?
Try changing api_manager.py to include a method for chat completion
import openai
from openai.error import InvalidRequestError
# ...
class ApiManager(metaclass=Singleton):
# ...
def create_chat_completion(
self,
messages: list, # type: ignore
model: str | None = None,
temperature: float = None,
max_tokens: int | None = None,
deployment_id=None,
) -> str:
cfg = Config()
if temperature is None:
temperature = cfg.temperature
try:
if deployment_id is not None:
response = openai.ChatCompletion.create(
deployment_id=deployment_id,
model=model,
messages=messages,
temperature=temperature,
max_tokens=max_tokens,
api_key=cfg.openai_api_key,
)
else:
response = openai.ChatCompletion.create(
model=model,
messages=messages,
temperature=temperature,
max_tokens=max_tokens,
api_key=cfg.openai_api_key,
)
except InvalidRequestError as e:
# Handle the specific error, e.g., if it's a token limit error
if e.code == "invalid_request" and "token limit" in e.message.lower():
# Implement your logic to reduce the length of messages or split them into smaller parts
# Modify messages or take appropriate action
pass
else:
# Handle other specific error cases or fallback to a general error handling strategy
pass
else:
if not hasattr(response, "error"):
logger.debug(f"Response: {response}")
prompt_tokens = response.usage.prompt_tokens
completion_tokens = response.usage.completion_tokens
self.update_cost(prompt_tokens, completion_tokens, model)
return response
Lets see if something like this would work.
Try changing api_manager.py to include a method for chat completion
import openai from openai.error import InvalidRequestError # ... class ApiManager(metaclass=Singleton): # ... def create_chat_completion( self, messages: list, # type: ignore model: str | None = None, temperature: float = None, max_tokens: int | None = None, deployment_id=None, ) -> str: cfg = Config() if temperature is None: temperature = cfg.temperature try: if deployment_id is not None: response = openai.ChatCompletion.create( deployment_id=deployment_id, model=model, messages=messages, temperature=temperature, max_tokens=max_tokens, api_key=cfg.openai_api_key, ) else: response = openai.ChatCompletion.create( model=model, messages=messages, temperature=temperature, max_tokens=max_tokens, api_key=cfg.openai_api_key, ) except InvalidRequestError as e: # Handle the specific error, e.g., if it's a token limit error if e.code == "invalid_request" and "token limit" in e.message.lower(): # Implement your logic to reduce the length of messages or split them into smaller parts # Modify messages or take appropriate action pass else: # Handle other specific error cases or fallback to a general error handling strategy pass else: if not hasattr(response, "error"): logger.debug(f"Response: {response}") prompt_tokens = response.usage.prompt_tokens completion_tokens = response.usage.completion_tokens self.update_cost(prompt_tokens, completion_tokens, model) return responseLets see if something like this would work.
Hi I have added your code to the api_manager.py file, but I am still encountering the following error. Please take a look:
Traceback (most recent call last):
File "
Yes, there's updates on my fork I'm working on and updates to the scripts that call api_manager within the LLM folder that need fixes... I think I updated my fork last night but I might not have pushed from my local to GitHub. I'll check later today when I get home.
