Yan Zhabin comments

Results 29 comments of


                                            Yan Zhabin

common: Generalized XML-style tool-call parsing with streaming support (GLM 4.5/4.6 + MiniMax M2 + SeedOSS + Kimi-K2 + Qwen3-Coder + Apriel-1.5 + Xiaomi-MiMo)

I'm looking forward to get this PR merged! @hksdpc255 Does it require a custom jinja template from the previous PR or it works good as is?

common: Generalized XML-style tool-call parsing with streaming support (GLM 4.5/4.6 + MiniMax M2 + SeedOSS + Kimi-K2 + Qwen3-Coder + Apriel-1.5 + Xiaomi-MiMo)

> I feel like this PR is a mixed bag. I like the core idea and I think it's high time we implemented something like that (as in a more...

common: Generalized XML-style tool-call parsing with streaming support (GLM 4.5/4.6 + MiniMax M2 + SeedOSS + Kimi-K2 + Qwen3-Coder + Apriel-1.5 + Xiaomi-MiMo)

> I’m fine with having experimental or even hacky parsers, as long as they live in separate files or modules. That way, they can be easily rewritten or replaced later...

Eval bug: crash on faulty tool call

Have this issue too.

Project still supported?

Looks like this project is dead. Last couple releases are broken (ROCm) and crashing on my machine (llama.cpp works great btw).

Option to disable VS Code's default sorting of suggestions

This ☝️

Eval bug: Crashes when model is loaded across a Vega VII card with Mi50s

Had the same issue

🚀 MAINTAINED FORK: inference-sh/llama-cpp-python – Active, Up-to-date, Contributors Welcome

While it's not available via pip yet: ```pyproject.toml dependencies = [ "llama-cpp-python", ] [tool.uv.sources] llama-cpp-python = { git = "https://github.com/inference-sh/llama-cpp-python.git", branch = "main" } ```

🚀 MAINTAINED FORK: inference-sh/llama-cpp-python – Active, Up-to-date, Contributors Welcome

@okaris > Please leave a comment with any PRs from here you would like me to merge. I saw somewhere PR for re-ranking support (idk how good is it). It...