mlx-swift-examples icon indicating copy to clipboard operation
mlx-swift-examples copied to clipboard

Add LLama 3.2 1B and 3B lightweight models descriptions

Open jsmp opened this issue 1 year ago • 1 comments

This adds ModelConfigurations for the new Llama 3.2 lightweight models with 1B and 3B parameters, and bumps the Xcode project dependency on swift-transformers so that it can use the new Sequence processor, required for Llama v3.2.

Llama 3.2 introduces two new lightweight versions with 1B and 3B parameters, which speeds the inference considerably when compared to the 3.1 8B version (on my M1 16Gb, it goes from 12 tokens/sec to 66 tokens/sec on the 1B model).

jsmp avatar Sep 27 '24 08:09 jsmp

Looks good! Can you run the formatting hook then we can merge it?

awni avatar Sep 28 '24 13:09 awni