After a few prompts, it stop responding to new ones

[franzkruhm]

Describe the bug

I am using pygmalion-6b (main, not sharded apparently)on a 3060 12gb with --listen --cai-chat --extensions gallery --auto-devices I use the web interface in firefox.

The colab version works fine and keeps responding to prompts But my local install stop after about 5, apparently as soon as the chat text box starts scrolling. I see the CUDA load spike briefly but then immediately drop and no response is generated. (0 tokens)

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

pygmalion-6b (main, not sharded apparently)on a 3060 12gb with --listen --cai-chat --extensions gallery --auto-devices web interface open in firefox

Screenshot

.

Logs

.

System Info

RTX 3060 12G, 16G RAM, Win10

[franzkruhm]

Seems limiting VRAM usage allows it to run for longer before it becomes an issue again.

[github-actions[bot]]

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.