mlx-swift-examples
mlx-swift-examples copied to clipboard
ml-explore
Examples using MLX Swift
The OpenELM instruct models don't seem to produce useful output with the current (empty) prompt template. I've tried other common templates, but they don't seem to work with these models....
I'm adding a repetitionPenalty to the GenerateParameters constructor. Regardless what values I set (I tried 0.5, 1, 1.2, 1.5), it crashes the program immediate as the evaluator runs. I was...
When I try to generate output with OpenElm 270M on a [2018 iPad Pro](https://support.apple.com/en-us/111974), the LLMEval demo app crashes with the following error: ``` Compiler failed to build request libc++abi:...
I added a configuration option for [OpenELM 1b Instruct 8-bit](https://huggingface.co/mlx-community/OpenELM-1_1B-Instruct-8bit) like so: ```swift public static let openelm1bInstruct8bit = ModelConfiguration( id: "mlx-community/OpenELM-1_1B-Instruct-8bit" ) { prompt in "\(prompt)" } ``` But when...
In `Libraries/LLM/Load.swift`, the tokenizer is first loaded. As part of this process, `config.json` is downloaded if it hasn't been downloaded already. Then that same file is included in `modelFiles` (which...
https://github.com/ml-explore/mlx-swift/pull/124 I haven't tested this with Phi 3.5 yet, but will do that tomorrow.
It seems there is a problem with memory allocation when processing longer prompts, I have used a prompt with around 3500 tokens in `LLMEval` and when processing the prompt, the...
The python `mlx_lm` implementation generates at ~101 tokens per second for `mlx-community/Phi-3-mini-4k-instruct-4bit`, whereas the swift code here generates at ~60 tokens per second. Here is my python implementation ```python from...
Hi, can you also write as example tool for fine tunings llm like gigabyte ai to utility https://www.gigabyte.com/Press/News/2201
When sorting entries on argSort(array, axis: -1, stream: .gpu), sys crashes -[MTLDebugComputeCommandEncoder _validateThreadsPerThreadgroup:]:1267: failed assertion `(threadsPerThreadgroup.width(929) * threadsPerThreadgroup.height(1) * threadsPerThreadgroup.depth(1))(929) must be
