Jack Wind comments

Repositories
Issues
Comments

Results 2 comments of


                                            Jack Wind

Handle longer prompt/generation

looks great!

[Feature Request] MLX_lm: Store KV cache of computed prompts to disk to avoid re-compute in follow-up runs

take a look at this open source library / repo: https://github.com/otriscon/llm-structured-output they have an implementation of a reusable KV cache for mlx. i've gotten it working - works surprisingly well!