Implement Exponential Backoff for ThrottlingException in InvokeModel Calls

[aazizisoufiane]

Issue: Encountering botocore.errorfactory.ThrottlingException when using invoke_model due to rate limits. AWS recommends exponential backoff for API retries.

Current Function: def _call_model(self, body: str) -> str: return self.predictor.invoke_model(modelId=self._model_name, body=body, accept="application/json", contentType="application/json")

Question: Would a PR implementing exponential backoff with the backoff library for this operation be welcome? Keen to contribute a solution to improve request reliability.

[arnavsinghvi11]

Hi @aazizisoufiane , a PR would definitely be welcome! Just curious though, which AWS model were you using for this? I'm assuming Anthropic since I only see that AWS LM configured with backoff but I know @drawal1 is working on a AWS LM refactor in #795 so feel free to check out if it's handled there or if it needs to be added.

[drawal1]

@arnavsinghvi11 PR #795 does not implement any special code for exponential backoff so it would be welcome. @aazizisoufiane can implement once PR #795 is merged.

just noticed there is a dsp/modules/aws_lm.py file that will also be obsolete after PR #795. Deleting it so there's no confusion...