replicate.exceptions.ModelError: 'Request 0 already exists.'

[jpiabrantes]

I made a prediction to a Mistral model and received the following error:

Traceback (most recent call last):
  File "/Users/joaoabrantis/Desktop/sakana/languagemodels/debate.py", line 45, in <module>
    correct = debate([models[n] for n in model_names], rounds, data)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joaoabrantis/Desktop/sakana/languagemodels/debate.py", line 20, in debate
    round_answers.append(run(model, ari_initial_prompt(problem)))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joaoabrantis/Desktop/sakana/languagemodels/llm_service.py", line 44, in run
    response = replicate.run(model.identifier, input=args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joaoabrantis/miniconda3/lib/python3.11/site-packages/replicate/client.py", line 148, in run
    return run(self, ref, input, **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joaoabrantis/miniconda3/lib/python3.11/site-packages/replicate/run.py", line 61, in run
    raise ModelError(prediction.error)
replicate.exceptions.ModelError: 'Request 0 already exists.'

How should I handle these errors?

[mattt]

Hi @jpiabrantes. A ModelError is an error returned by the model, so there's nothing to be done from the client.

Could you share a link to the model — or better yet, to a prediction that failed with that error?

[neverabsolute]

Hey, seeing identical issues here. It's a bit sporadic, but when using mistralai/mistral-7b-instruct-v0.2, or any other model, with the same prompt and input, I will see this.

I suspect simply raising for retry on my end will help prevent this, but I wonder what the root cause is here?

[mattt]

The latest release of the Python client library (0.29.0) adds a prediction field to ModelError to help debug and respond to failures. If you're still seeing these issues intermittently, you can

import replicate
from replicate.exceptions import ModelError

try:
  output = replicate.run("mistralai/mistral-7b-instruct-v0.2", { "prompt": "..." })
except ModelError as e
  if "Request 0 already exists" in e.prediction.error:
    # (add retry logic here)

  print("Failed prediction: " + e.prediction.id)