Extract openai API calls and retry at lowest level

[collijk]

trafficstars

Background

Fourth PR in the chain to organize the LLM interaction. See:

• #3191
• #3436
• #3486

In a previous PR, I extracted a retry decorator from the embedding calls. This PR uses the same retry decorator on an extracted call to create chat completions from the OpenAI API.

Changes

• APIManager now only manages API budgets. Will be renamed and updated to use LLM structs in the next PR.
• autogpt.llm.llm_utils.retry_openai_api moved to autogpt.llm.providers.openai.retry_api
• Low level API methods for creating embeddings and chat completions have been extracted to autogpt.llm.providers.openai and have been appropriately decorated to consistently handle API failure cases.
• autogpt.llm.llm_utils.create_chat_completion refactored to use low-level wrapped api calls and update the budget manually.
• Move retry handler tests from tests.unit.test_llm_utils to tests.unit.test_openai

Documentation

N/A, this is primarily code motion

Test Plan

N/A this is primarily code motion and the code in question has unit tests.

PR Quality Checklist

• [X] My pull request is atomic and focuses on a single change.
• [X] I have thoroughly tested my changes with multiple different prompts.
• [X] I have considered potential risks and mitigations for my changes.
• [X] I have documented my changes clearly and comprehensively.
• [X] I have not snuck in any "extra" small tweaks changes

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)	Visit Preview		Jun 12, 2023 0:42am

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[codecov[bot]]

Codecov Report

Patch coverage: 86.31% and project coverage change: +0.38 :tada:

Comparison is base (49d1a5a) 70.74% compared to head (5c7b222) 71.13%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3696      +/-   ##
==========================================
+ Coverage   70.74%   71.13%   +0.38%     
==========================================
  Files          72       72              
  Lines        3504     3509       +5     
  Branches      556      556              
==========================================
+ Hits         2479     2496      +17     
+ Misses        856      843      -13     
- Partials      169      170       +1     

Impacted Files	Coverage Δ
autogpt/app.py	48.35% <0.00%> (-1.09%)	:arrow_down:
autogpt/llm/utils/__init__.py	49.20% <57.14%> (-12.54%)	:arrow_down:
autogpt/llm/providers/openai.py	93.50% <93.15%> (-6.50%)	:arrow_down:
autogpt/llm/api_manager.py	100.00% <100.00%> (+3.57%)	:arrow_up:
autogpt/llm/base.py	92.77% <100.00%> (+0.17%)	:arrow_up:
autogpt/memory/vector/utils.py	83.87% <100.00%> (-1.85%)	:arrow_down:

... and 3 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[p-i-]

This is a mass message from the AutoGPT core team. Our apologies for the ongoing delay in processing PRs. This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to: https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[lc0rp]

James, @collijk - I know you've got your head down. Is this worth trying to put in 0.3.2? Is anyone else available to resolve conflicts?

[lc0rp]

Seems to me, this could still be valuable even after the re-arch. Marking for maintainer-review

[vercel[bot]]

Deployment failed with the following error:

Resource is limited - try again in 3 minutes (more than 100, code: "api-deployments-free-per-day").

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[Pwuts]

Needs check if still relevant and rebase if so

[ntindle]

Needs rebasing

[p-i-]

What's going on with:

Some checks were not successful Docker CI / test (pull_request) Cancelled after 327m Python CI / test (3.10) (pull_request_target) Cancelled after 329m

Are we racking up costs here?

Should we have a safeguard to abort after say 300s?

[vercel[bot]]

Deployment failed with the following error:

Resource is limited - try again in 19 minutes (more than 100, code: "api-deployments-free-per-day").

[github-actions[bot]]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[vercel[bot]]

Deployment failed with the following error:

Resource is limited - try again in 18 minutes (more than 100, code: "api-deployments-free-per-day").

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[vercel[bot]]

Deployment failed with the following error:

Resource is limited - try again in 17 minutes (more than 100, code: "api-deployments-free-per-day").

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[vercel[bot]]

Deployment failed with the following error:

Resource is limited - try again in 14 minutes (more than 100, code: "api-deployments-free-per-day").

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[vercel[bot]]

Deployment failed with the following error:

Resource is limited - try again in 8 minutes (more than 100, code: "api-deployments-free-per-day").

[github-actions[bot]]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[vercel[bot]]

Deployment failed with the following error:

Resource is limited - try again in 30 seconds (more than 100, code: "api-deployments-free-per-day").