Make Auto-GPT aware of its running cost

[Vwing]

Background

This pull request addresses issue #6, which aims to make Auto-GPT aware of its running cost. The changes centralize all OpenAI Chat Completion calls through a single method that counts tokens and adds to a dollar amount based on the model used. The total dollar amount spent is constantly displayed to the user and is shown to the AI as well.

The user is also asked to provide an optional API budget dollar amount. If they enter nothing, there is no monetary limit, but if they define a budget then the AI will be told to shut down gracefully once it has come within 1 cent of its limit, and to shut down immediately once it has exceeded its limit. The AI is always aware of its initial and remaining budget.

Changes

• Centralized chat completion API calls through ApiManager class in api_manager.py
• Removed openai imports from other files to prevent accidental API calls
• Updated AIConfig class to include an optional api_budget parameter
• Modified chat_with_ai function to include api_budget and display remaining budget to the AI
• Updated browse.py, call_ai_function.py, chat.py, and main.py to use the centralized ApiManager
• Added input for API budget to load_variables and construct_prompt functions in main.py

Documentation

There are Docstrings and in-code comments documenting things where necessary

Test Plan

I have tested this feature in the following ways:

• Various different prompts were used, assigning Auto-GPT to various different tasks, and it did not encounter any more errors than before the change
• I put an empty string for the budget, and the AI proceeded as if it had infinite budget
• I put a dollar amount for the budget, and the AI proceeded as if it had a limited budget, hurrying itself up as the budget depleted
• The AI shuts itself down once the budget is exceeded
• The total running cost is being printed to the console for the user to see upon every chat completion, and the amount makes sense for the models used

PR Quality Checklist

• [x] My pull request is atomic and focuses on a single change.
• [x] I have thoroughly tested my changes with multiple different prompts.
• [x] I have considered potential risks and mitigations for my changes.
• [x] I have documented my changes clearly and comprehensively.
• [x] I have not snuck in any "extra" small tweaks changes

[jbh2155]

I hope this gets added - it's pretty fundamental information for Auto-GPT, especially Entrepreneur-GPT (the default agent).

[Vwing]

The change scope is too big and it needs to be split. So I suggest just to extract the api_manager from the code first, preserving the current behavior. And then change the behavior in a separate PR.

Sure! I could make a new PR with just the first two commits. Those only deal with adding the api_manager, routing chat completions through it, and printing the total running cost to the console.

e168ab0 8492a7f

EDIT:

oops, I lied. It also prints the total running cost to the AI. Should I remove that part in my new pull request?

        create_chat_message(
            "system", f"Your current running cost is ${api_manager.get_total_cost():.3f}"),

[nponeccop]

@Torantulino we need you to approve the plan

[Vwing]

@nponeccop @Torantulino Shall I implement the plan detailed above? In my new PR, I can also update the code with the latest changes to master.

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[Vwing]

I'll take those bot messages as a sign I should stay in this pull request, merge in master, and get everything working with the latest changes? Okay, will do!

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[rob-luke]

I am so pleased someone created this PR, thanks @Vwing. This does not track the cost of the embedding calls. While this could be it be added in a future PR, it may confuse end users when their bill is larger than they expect. I am happy to contribute this extra feature if you want, just let me know.

[Vwing]

Sure, @rob-luke. That would be much appreciated!

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[Vwing]

@rob-luke Heads up, I merged in master to resolve the conflicts.

[Pwuts]

Almost there, please fix the linting errors 🥁

[Vwing]

@Pwuts done!

[rob-luke]

I dont want to block this from getting merged, so I will just create a follow up PR once this is merged

[rob-luke]

I added the tracking of embedding costs and opened a PR in to this branch here: https://github.com/Vwing/Auto-GPT/pull/1 But if you merge this branch as is, then I will just open a fresh PR in to master of this repo later. Thanks again for the great package and for this PR.

[Vwing]

I merged in your changes, @rob-luke, they look good! Just make sure next time to run the linter

[Vwing]

@BillSchumacher I don't know if you've started reviewing this PR yet, but just to let you know I merged in @rob-luke's changes so it tracks embedding costs.

[rob-luke]

Thanks Vwing, and sorry for missing the linting. Thanks for fixing my errors

[ntindle]

Can you update from the latest? theres some changes to the pipelines that need to run @Vwing

[Vwing]

Yep! I'm on it.

[ntindle]

Give it a bit, you’ll have to do it twice if not

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[Vwing]

Okay, I screwed up and merged too soon 😅

I'm going to try to clean up my history. I'll be a moment...

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[ntindle]

Heads up this may need scheduled in

[Vwing]

Oops, some tests failed. I can fix that...

[Vwing]

Hi @ntindle, I notice that the test is failing because it's using the "gpt2" model, which isn't an option for OpenAI chat completions. I'm wondering how we should proceed in this case. Should I check for invalid models and fail quietly, or do you think it's better to update the test to use a valid model? Looking forward to your guidance, thanks!