jonmach
jonmach
I just tried to build (Mac Silicon - M2) using your Dockerfile and it fails in step 6/8 on the following line: `g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17...
To be honest, I only need this to be an embedding server. I'll be using LM Studio as my LLM server. Would you have a recommendation on what the Dockerfile...
Thanks for coming back on this - In the end, I gave up on LocalAI. It was just too bulky (34GB) for the small area of functionality I need, and...
I think you're correct that Docker doesn't support metal. For my purposes (dev/test etc.) I needed to be able to fail fast. CPU was good enough since it was going...
I can't comment on the quality of the container design. In my mind, these things sometimes take time, and as long as it functionally works, then improvements can be made....
This seems to work already
I didn't realise it wasn't supported. I just loaded it up and it worked. See for example this using /info when using localhost:8080/info: A test of a vector encoding also...
> @jonmach Hi, could you please share how do you start TEI with bge-m3? I'm having trouble starting the service. Sure - this is all I do, which gives me...
Same problem for me