AutoGPT icon indicating copy to clipboard operation
AutoGPT copied to clipboard

Huggingface retry generate_image with delay

Open primaryobjects opened this issue 2 years ago • 19 comments

Background

The command generate_image with huggingface may return an error response due to the service queuing the request. This results in the image failing to generate. A delay may be invoked and the request retried, in order to return a successful image.

cannot identify image file <_io.BytesIO object at 0x7f6eac20ef40>
b'{"error":"Model CompVis/stable-diffusion-v1-4 is currently loading","estimated_time":20.0}'

Changes

  • Check for error in response from huggingface and retry with delay up to 10 times.
  • Delay in seconds is obtained from huggingface response.

Documentation

  • No additional documentation required for users.

Test Plan

  • Tested locally by running with generate_image command.
  • Added unit tests for successful and failure cases. See Report.

PR Quality Checklist

  • [X] My pull request is atomic and focuses on a single change.
  • [X] I have thoroughly tested my changes with multiple different prompts.
  • [X] I have considered potential risks and mitigations for my changes.
  • [X] I have documented my changes clearly and comprehensively.
  • [X] I have not snuck in any "extra" small tweaks changes

primaryobjects avatar Apr 21 '23 01:04 primaryobjects

Do me a solid and update this so we can merge

ntindle avatar Apr 23 '23 02:04 ntindle

Codecov Report

Patch coverage: 88.88% and project coverage change: -7.32 :warning:

Comparison is base (06e81b7) 49.70% compared to head (6501f38) 42.38%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2745      +/-   ##
==========================================
- Coverage   49.70%   42.38%   -7.32%     
==========================================
  Files          64       64              
  Lines        3026     3048      +22     
  Branches      507      510       +3     
==========================================
- Hits         1504     1292     -212     
- Misses       1402     1689     +287     
+ Partials      120       67      -53     
Impacted Files Coverage Δ
autogpt/commands/image_gen.py 58.66% <88.88%> (+32.25%) :arrow_up:

... and 14 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov[bot] avatar Apr 23 '23 16:04 codecov[bot]

Code coverage added for all cases here.

python -m pytest .\tests\test_image_gen.py::TestImageGenFailure
== test session starts ==
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)                                 [100%] 
plugins: anyio-3.6.2, asyncio-0.21.0, benchmark-4.0.0, cov-4.0.0, integration-0.2.3, mock-3.10.0, vcr-1.0.2                                                                                               ============= 
asyncio: mode=strict
collected 3 items

tests\test_image_gen.py ... [100%]

== 3 passed in 0.63s ====
python -m pytest .\tests\test_image_gen.py::TestImageGen -k huggingface
== test session starts ==
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
plugins: anyio-3.6.2, asyncio-0.21.0, benchmark-4.0.0, cov-4.0.0, integration-0.2.3, mock-3.10.0, vcr-1.0.2
asyncio: mode=strict
collected 3 items / 2 deselected / 1 selected

tests\test_image_gen.py . [100%]

== 1 passed, 2 deselected in 59.96s ==

primaryobjects avatar Apr 23 '23 20:04 primaryobjects

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions[bot] avatar Apr 24 '23 13:04 github-actions[bot]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

github-actions[bot] avatar Apr 24 '23 14:04 github-actions[bot]

Codecov Report

Patch coverage: 88.88% and project coverage change: +0.57 :tada:

Comparison is base (c1cd54d) 62.18% compared to head (03cb399) 62.75%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2745      +/-   ##
==========================================
+ Coverage   62.18%   62.75%   +0.57%     
==========================================
  Files          73       73              
  Lines        3345     3367      +22     
  Branches      484      487       +3     
==========================================
+ Hits         2080     2113      +33     
+ Misses       1118     1107      -11     
  Partials      147      147              
Impacted Files Coverage Δ
autogpt/commands/image_gen.py 69.33% <88.88%> (+33.48%) :arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov[bot] avatar Apr 25 '23 04:04 codecov[bot]

👋 jerrod from Codecov here, we are looking at this PR to see if we can figure out what is causing the unexpected coverage change

jerrodcodecov avatar Apr 25 '23 16:04 jerrodcodecov

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions[bot] avatar Apr 26 '23 17:04 github-actions[bot]

This is a mass message from the AutoGPT core team. Our apologies for the ongoing delay in processing PRs. This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to: https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

p-i- avatar May 05 '23 00:05 p-i-

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
docs ⬜️ Ignored (Inspect) Visit Preview May 16, 2023 2:57pm

vercel[bot] avatar May 12 '23 06:05 vercel[bot]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

github-actions[bot] avatar May 12 '23 06:05 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar May 12 '23 06:05 github-actions[bot]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions[bot] avatar May 12 '23 06:05 github-actions[bot]

Fixed the tests partially by removing them 😅

If you can add a test that's appropriate

ntindle avatar May 12 '23 07:05 ntindle

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions[bot] avatar May 13 '23 20:05 github-actions[bot]

Deployment failed with the following error:

Resource is limited - try again in 1 hour (more than 100, code: "api-deployments-free-per-day").

vercel[bot] avatar May 14 '23 01:05 vercel[bot]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

github-actions[bot] avatar May 14 '23 01:05 github-actions[bot]

Deployment failed with the following error:

Resource is limited - try again in 56 minutes (more than 100, code: "api-deployments-free-per-day").

vercel[bot] avatar May 14 '23 01:05 vercel[bot]

Deployment failed with the following error:

Resource is limited - try again in 34 minutes (more than 100, code: "api-deployments-free-per-day").

vercel[bot] avatar May 14 '23 13:05 vercel[bot]