Jesse
I am in the process of encapsulating this into an MCP server. I think it's mostly done: https://github.com/createthis/diffcalculia_mcp EDIT: Open Hands AI sees the tools! Pretty cool!
I've uploaded the bf16 here, if anyone needs it: https://huggingface.co/createthis/DeepSeek-V3.2-Exp-bf16/tree/main
```bash (/data/conda-envs/llama_cpp_cu128) jesse@larry:~/llama.cpp$ python convert_hf_to_gguf.py --outtype f16 --split-max-size 50G --outfile /data2/DeepSeek-V3.2-Exp-GGUF/bf16 /data2/DeepSeek-V3.2-Exp-bf16/ INFO:hf-to-gguf:Loading model: DeepSeek-V3.2-Exp-bf16 WARNING:hf-to-gguf:Failed to load model config from /data2/DeepSeek-V3.2-Exp-bf16: The checkpoint you are trying to load has...
Quick update ... I woke up early this morning and did my best to figure out the tensor mappings. I've updated my branch: https://github.com/createthis/llama.cpp/pull/9/files Using this command the GGUF bf16...
FYI: Aider polyglot benchmarks indicate the VLLM code is underperforming vs deepseek's own stack. See https://discord.com/channels/1131200896827654144/1422153923493498902/1423627025062690816 for more info. We should be careful when cargo culting VLLM's code and see...
With the last two commits I've raised the tensor count error from `expected 1391, got 1086` to `expected 1391, got 1330`. Seems like progress. Still missing something though. I deleted...
Haha. Wow. As of [bb1f4e4](https://github.com/createthis/llama.cpp/pull/9/commits/bb1f4e454a8de7966a78978832aae7fc9125e1fe) it is inferring on my system locally. Performance is terrible, and the last three or four commits were entirely authored by DeepSeek V3.1-Terminus, so I...
Here's the Q4_K_M GGUF: https://huggingface.co/createthis/DeepSeek-V3.2-Exp-GGUF/tree/main/q4_k_m EDIT: I tried to upload the bf16 GGUF again, but I was hit with this gem: > Your storage patterns tripped our internal systems! Please...
I deleted some old models. Here's the bf16 GGUF: https://huggingface.co/createthis/DeepSeek-V3.2-Exp-GGUF/tree/main/bf16
I turned off the logging statement. Performance is actually just as good as V3.1-Terminus on my machine: ```bash echo 0 | sudo tee /proc/sys/kernel/numa_balancing echo 3 | sudo tee /proc/sys/vm/drop_caches...