Toolio
Toolio copied to clipboard
OoriData
GenAI & agent toolkit for Apple Silicon Mac, implementing JSON schema-steered structured output (3SO) and tool-calling in Python. For more on 3SO: https://huggingface.co/blog/ucheog/llm-power-steering
I tried to run some models with Toolio in server mode. I'm running the server in DEBUG to see all the logs. When prompting a model in both TypingMind and...
In supporting concurrent requests, we won't at first assume concurrent inference capability at the model weight stage. This would require us to mutex the LLMs. We'll want control over the...
Server: ```sh toolio_server --model=mlx-community/Phi-3-mini-128k-instruct-4bit ``` Client: ```sh toolio_request --apibase="http://localhost:8000" --prompt='What is the average airspeed of an unladen swallow?' ``` You can run the above any number of times, but as...
This is due to an [upstream change in mlx_lm](https://github.com/ml-explore/mlx-examples/commit/561dcf5643ecced3be9bf053499e0e2ea51e8686). While investgating & fixing I also caught & applied [based on this earlier patch](https://github.com/ml-explore/mlx-examples/commit/69181e00584976f717c27f90c7e009e70cc1b0bf). [Fixed in main](https://github.com/OoriData/Toolio/commit/12e74738d64110bb77839ee231ac5bc058850072). Will release Toolio 0.3.1...
In adding support for Gemma models ( #6 ) I set up model flags per model type, including one for whether or not the system role is supported, `model_flag.NO_SYSTEM_ROLE`. Then...
Follow in the [footsteps of OgbujiPT](https://github.com/OoriData/OgbujiPT/issues/90).
Grabbed a subset of [this wiki thingie](https://www.reddit.com/r/ListOfSubreddits/wiki/listofsubreddits), trimmed a few of the more salacious bits (I'm not prude, but also not in the mood to exclude prudes). [subreddits_raw_struct.html.zip](https://github.com/user-attachments/files/17626569/subreddits_raw_struct.html.zip) Build on...
I worked with Claude Code on some enhancements, but don't worry. I'm doing my engineering on it as well, just to be sure 😉. Here is Claude's own summary: >...