Stephan Janssen

I'm currently copying the directories manually from .cache/huggingface/hub/models directories using AirDrop to the other machines. This works and is obviously faster when downloading from huggingface. models--mlx-community--Meta-Llama-3.1-70B-Instruct-4bit models--mlx-community--Meta-Llama-3.1-8B-Instruct-4bit

Thanks for reporting, will have a closer look.

Great, I have the same issue on my Mac so I can debug it further...

It's actually a Langchain4J MCP Bug, which I've fixed. You can manually install the fix and try it out from https://drive.google.com/file/d/1XCbngvy7gyRn1q9a4tHRPmGPFlIGA-OA/view?usp=share_link Created and issue on the Langchain4J project @ https://github.com/langchain4j/langchain4j/issues/3118

Great! Out of interest, which LLMs are you using with this MCP ?

Maybe the latest DeepSeek will do better? https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/blob/main/README.md

Having a compliant OpenAI (chat) REST API would be amazing. This would allow many tools (including LangChain4J) to integrate without any extra code with Llama3.java See also https://platform.openai.com/docs/api-reference/chat

Agreed, similar to what I've done as an experiment @ https://github.com/stephanj/Llama3JavaChatCompletionService But then better 😂

We currently do allow the user to add the project BEFORE the actual user prompt is provided. Keeping a real-time counter of tokens for both project context and the prompt...

Adding "--platform linux/amd64" to the docker command gets rid of the warning: ``` docker run --platform linux/amd64 -e GRADIO_SERVER_NAME=0.0.0.0 -e GRADIO_SERVER_PORT=7860 -p 7860:7860 -it --rm taprosoft/kotaemon:v1.0 ``` However the next...