AT
AT
These other servers would all expose OpenAI compatible API then? Do you have a specific example you're testing with?
I don't know what this means. Would need a UI image to even begin to evaluate.
This will require extensive changes to the GUI as well. It has been agreed that the GUI changes will come first to provide a UI for the current multimodel upstream.
This is caused by an inability to switch on/off GPU mode per model instance. Right now we do it on per model type. We need to fix vulkan backend so...
This is because you don't have enough VRAM available to load the model. Yes, I know your GPU has a lot of VRAM but you probably have this GPU set...
You should be able to use partial offloading now to load some number of the layers of the model into VRAM even for 16GB models. I'm going to wait a...
> Although it may be possible for some users to mitigate this issue with partial offloading, it is still an issue - people should be able to fully offload models...
Does this have a vulkan driver available on Apple/Mac? That's the most important question...
@cebtenzzre this is quite old. anything to salvage from this or close?
Is this out of date and to be closed?