Zeng Lingyu
Zeng Lingyu
这个只针对windows和macos生效，我正准备直接融合到dify的docker-compose.yml里面进行访问，我是通过将dify穿透到公网进行访问的，内部访问似乎有点问题。
I use 8 macs
 it show like this
mac mini(10.25.0.2-7) can not use internet, so show: ``` "~/project /exo/.venv/lib/python3.12/ site-packages/aiohttp/conn ector.py", line 1341, in _create_direct_connection raise ClientConnectorDNSError(re q.connection_key, exc) from exc aiohttp.client_exceptions. ClientConnectorDNSError: Cannot connect to host huggingface.co:443...
```shell libc++abi: terminating due to uncaught exception of type std::runtime_error: [METAL] Command buffer execution failed: Caused GPU Timeout Error (00000002:kIOGPUCommandBufferCallbackErrorTimeout) zsh: abort exo --node-id=$NODE_ID --node-host=$CURRENT_HOST --discovery-module=udp ``` ____ After enabling...
> Indeed, I do believe your mac cluster has enough capability to run the q4 mlx version (BTW, I just watched the real instance that one mac with M4 Max...
> Sounds good! 老哥你现在速度多少啊？这样8台跑4位mlx版的r1是不是有点浪费，我感觉你跑原生671B都可以。 fp8 不支持的mlx没有加速
原生内存不够的，考虑到对话过程中有kvcache
libc++abi: terminating due to uncaught exception of type std::runtime_error: [METAL] Command buffer execution failed: Caused GPU Timeout Error (00000002:kIOGPUCommandBufferCallbackErrorTimeout) zsh: abort exo --node-id=$NODE_ID --node-host=$CURRENT_HOST --discovery-module=udp —————— I do not kown...
都是最新的代码，mlx版本都应该是一致的。反正每次运行都有两台机器出现这个报错，然后退出。似乎在运行时候会哈希校验，必须从huggingface下载，从其他镜像下载就运行不了，报md5校验不一样