ControlFlow icon indicating copy to clipboard operation
ControlFlow copied to clipboard

Make retries first class citizens in ControlFlow

Open discdiver opened this issue 1 year ago • 0 comments

Enhancement Description

The ability to retry on task failure-with options for retry delays including exponential backoff-would be useful functionality to interface from ControlFlow directly. It's possible with ControlFlow and Prefect, but requires knowledge of Prefect.

Use Case

Retrying API calls is a common need with LLMs.

Proposed Implementation

Adding directly to ControlFlow task keyword arguments might be nice.

```python
task = cf.Task(
    objective="Write a poem about the provided topic",
    instructions="Write four lines that rhyme",
    context={"topic": "AI"},
    retries=4
)

Context from Prefect Community Slack:

quick question: in ControlFlow is there a way to apply exponential backoff to avoid rate limiting? I got a few error messages after the agents running for a few minutes: ...

Solution from @zzstoatzz:

i would just wrap the thing you want to retry in a prefect task (since prefect is already installed when you use cf)

from prefect.tasks import exponential_backoff from prefect import task

t = task(lambda: 1/0)

with_retries = t.with_options(retries=10, retry_delay_seconds=exponential_backoff(3))

you can do this inlined task(**options)(some_callable) syntax or the more traditional decorator syntax:

python @task(retries=..., ...) def some_callable(): ...

discdiver avatar Dec 05 '24 18:12 discdiver