David Koski

I don't think this is going to be possible on iPad -- you can [read about it here](https://medium.com/@blazejsleboda/using-3rd-party-frameworks-in-swift-playgrounds-for-ipad-or-macos-72a3fd210a2a). In order to do this we would need to put all of...

I wonder if you could set `SWIFTPM_BUNDLE` when building libmlx? When set it will use NSBundle.mainBundle followed by `SWIFTPM_BUNDLE` to try to resolve the path. Then it would just be...

3.9.0 -> 3.9.1

> Using the chat templates with Llama 3.2 will result in an error until `none` and `tojson` are implemented in the Jinja package: [maiqingqiang/Jinja#4](https://github.com/maiqingqiang/Jinja/pull/4) OK, should we hold off on...

I don't know what kind of fine tuning they do, but check out: - https://github.com/ml-explore/mlx-swift-examples/blob/main/Tools/llm-tool/README.md#lora That is fine tuning using LoRA (https://arxiv.org/abs/2106.09685)

Closing issue, but if you have more information please reopen or file a new issue!

I will investigate this

I can see two things that look like they are contributing here: - JIT startup costs -- the first time I ran the swift version I saw ~60 tokens per...

TASKs: - port `StreamingDetokenizer` - use `mx.async_eval(y)` to pipeline the generation - look at `KVCache` from the python side as well We can do these in that order as the...

The performance should be roughly the same as python now, though I found both of them to be a little noisy in the measurement. See #109