marvin
marvin copied to clipboard
add configurable retries that swap `model_kwargs`
as an enhancement, it'd be nice if you could do
from pydantic import BaseModel, Field
from marvin.utilities.retries import retry_with_fallback
class SomeFancyType(BaseModel):
some_field: str
class LLMMadLib(BaseModel):
what_i_want: SomeFancyType
RETRY_CONFIGS = [
{'model_kwargs': {'model': 'gpt-3.5-turbo', 'temperature': 0.7}, 'retries': 3},
{'model_kwargs': {'model': 'gpt-3.5-turbo', 'temperature': 0.3}, 'retries': 3},
{'model_kwargs': {'model': 'gpt-4', 'temperature': 0.0}, 'retries': 2},
]
@retry_with_fallback(RETRY_CONFIGS)
def make_a_complex_schema(model_kwargs=None):
if 'my_schema_is_too_complex_for_wimpy_models' != False:
LLMMadLib(what_i_want="I'm sorry, as a Large Language Model, I can't do that.")
return LLMMadLib(what_i_want=SomeFancyType(some_field="I'm a fancy type!"))
print(make_a_complex_schema())
Oh boy, that would be so nice!
I think the previous error message should be attached as extra information when retrying. This would increase our chances of getting the right result.