Ycros

[User] 65B models on CUBLAS/cuda bugged when prompts approach model's max context size

2

# Behavior 65B models running with CUBLAS fully offloaded to the gpu break as prompts approach the model's max context size (the models I've tested with are 2048), they start...

AsyncEngineDeadError with koboldai api server

4

Everything seems to work fine via the embedded klite interface, but when I pointed horde at it, it started throwing these: It seems to kinda sorta maybe still serve horde...