Jason

I do have mtail running on windows aswell but im not sure on how it would incorporate with the docker compose files. Any useful guides you could point me to...

@pikolos how did you run the mtail in windows, could you tell me the steps.

Would love to see this implemented. This is the best small llm so far we have seen.

it would be quite an interesting challenge, vllm supports concurrency natively so some things would surely be a plus.

`livepeer[7148]: E0211 10:20:44.236142 7148 segment_rpc.go:128] manifestID=b48a3a03-736a-456a-a65e-09db797ff44d seqNo=3931 orchSessionID=19b73b91 clientIP=185.59.221.179 sender=0xc3c7c4C8f7061B7d6A72766Eee5359fE4F36e61E Could not read request body - err="stream error: stream ID 5; CANCEL"` `livepeer[7148]: [h264_cuvid @ 0x7fe8ca813640] ctx->cvdl->cuvidParseVideoData(ctx->cuparser, &cupkt) failed ->...

@vaclcer is it possible for you to create a guide or instructions on how to it. Would love to try out the new embedding model.

@FranckUltima you can use proxyd right now to have a backup RPC, I have multiple RPC backends just incase the main RPC goes down, it automatically switches to the other...