eval-dev-quality icon indicating copy to clipboard operation
eval-dev-quality copied to clipboard

Rethink retry logic for LLM Providers

Open Munsio opened this issue 1 year ago • 0 comments

This image shows the uptime of the RWKV v5 World 3B model: image

This model is so long down that the retry logic we are currently using does not really work, but waiting for the model to be available again would only stretch out the evaluation runs artificially

Munsio avatar Aug 01 '24 08:08 Munsio