David Young
David Young
Having same issue here @md5 @rokcarl. Any luck fixing it so far @rokcarl?
@md5 thanks for getting back to me so soon. I was using Chrome on OS X. I have two docker containers with two different VIRTUAL_HOST's set. One is a staging...
@md5 upon restarting chrome and clearing settings blog.example.io still redirects to staging.example.io but www.example.io does not and shows a 503.
@md5 that makes sense actually. When I say redirects, I mean the wrong backend container servers the request. How would I go about fixing this with an ssl config? It...
Awesome! Glad to help! Appreciate fast merge!
Whilst debugging, I attempted to add --trust-remote-code and build from source without any improvement, in case you think I just needed the latest mixtral fix in commit.
Turns out it isn't specifically Mistral/Mixtral, but more so the way the communication happens across GPUs.
For anyone that has gone down a rabbit hole with this, what was missing was --ipc=host on docker run. Finally!
Sorry for what is likely a silly question, but does this have an impact on KV cache size when using full offload with CUDA? Because that would be very appealing...
I seem to be facing some overflow issues with this PR, which I don't face with mainline, possibly due to `tp=16` - 16x3090 gpus: ...