Eric Buehler
Eric Buehler
I agree, as it would be more cohesive and we would need a wrapper for `ort` anyway. @LaurentMazare, do you think this would be better suited as an internal or...
Looks like it is its entity, so its types and API would not match Candle's. Perhaps we could base a custom implementation off of wonnx. @maxwellflitton, what are your thoughts...
> Is there an issue getting changes from the fork upstreamed? Ideally you don't need to maintain a fork, especially if it starts to diverge and become a hassle to...
Hi @mattia-chiozzi! Can you please try #1706 to see if it resolves the issue? To do that: ``` git pull git checkout fix_mcp_server_hang ``` And then rebuild.
Hi @mattia-chiozzi! I can't reproduce this with the following MCP config: ``` { "servers": [ { "name": "Filesystem Tools", "source": { "type": "Process", "command": "npx", "args": ["@modelcontextprotocol/server-filesystem", "."] } }...
> @EricLBuehler **thank you!** > > The initial metrics points look great. Do we have the ability to see allocation/free sizes from those call parameters or is that something which...
Hi @sempervictus! Something I've explored is offloading the KV prefix cache to a vector DB. Will take a look at this.
Hi @sempervictus! What projects would you have in mind that could benefit from such a crate? With regards to limitations, we are working on the "memory bloat" actively. I feel...
Hi @celsowm - this looks really good at first glance! Did you do any testing of it?
> Not yet, sorry, very busy week :( No problem, will take a look!