Prediction failed for an unknown reason. It might have run out of memory? (exitcode 2)

[cudanexus]

When running locally the cog predict -i it working fine but when pushed on replicate it giving this error Traceback (most recent call last): File "/root/.pyenv/versions/3.8.16/lib/python3.8/site-packages/cog/server/runner.py", line 334, in predict return _predict( File "/root/.pyenv/versions/3.8.16/lib/python3.8/site-packages/cog/server/runner.py", line 370, in _predict for event in worker.predict(input_dict, poll=0.1): File "/root/.pyenv/versions/3.8.16/lib/python3.8/site-packages/cog/server/worker.py", line 135, in _wait raise FatalWorkerException( cog.server.exceptions.FatalWorkerException: Prediction failed for an unknown reason. It might have run out of memory? (exitcode 2) any idea @anotherjesse @bfirsh

[arnavmehta7]

Same issue 🤔

[Tariqh23]

Have you got any solution?

[Tariqh23]

@anotherjesse @bfirsh

[arnavmehta7]

No solution

[wolfpixels]

+1

[mattt]

Hi @cudanexus. Can you please share the name of the model on Replicate?

[AnushreeBannadabhavi]

+1

[mattt]

@AnushreeBannadabhavi @wolfpixels @arnavmehta7 @Tariqh23 If you're seeing the same behavior, could you please share the name of the Replicate model you're using?

[Muhammadfawad267]

When running locally the cog predict -i it working fine but when pushed on replicate it giving this error Traceback (most recent call last): File "/root/.pyenv/versions/3.8.16/lib/python3.8/site-packages/cog/server/runner.py", line 334, in predict return _predict( File "/root/.pyenv/versions/3.8.16/lib/python3.8/site-packages/cog/server/runner.py", line 370, in _predict for event in worker.predict(input_dict, poll=0.1): File "/root/.pyenv/versions/3.8.16/lib/python3.8/site-packages/cog/server/worker.py", line 135, in _wait raise FatalWorkerException( cog.server.exceptions.FatalWorkerException: Prediction failed for an unknown reason. It might have run out of memory? (exitcode 2) any idea @anotherjesse @bfirsh

[ZYLIM0702]

do you get the solution. Can you share with me ? Looking forward. Thanks

[makaronide]

https://replicate.com/findix/sd-scripts @mattt The error occurs on this model.

[Archcollege]

i have the same problem.https://replicate.com/archcollege/voice

[notedit]

same here，when i run cog predict in local.

[quocanh34]

Does anyone have solutions?

[notedit]

I came to this and fix it, maybe your predict func came to some exception that does not be catched.

[arnavmehta7]

I've not worked on cog from a lot of time, but this might help someone:

I was able to run it locally as I had better configurations locally than the VM on which I was trying to deploy. I will suggest reducing the model size or increasing the memory of the machine on which you are trying to deploy.

Additionally try to add a try-except block and do following everywhere in the exception block:

import traceback
print(traceback.format_exc())

[Chyrets]

has anyone solved the problem?

[emcmanus]

Try deploying on more powerful hardware. At least in my case, Cog really did run out of memory.