mlx-swift-examples icon indicating copy to clipboard operation
mlx-swift-examples copied to clipboard

Error when running OpenELM 1b Instruct 8-bit

Open DePasqualeOrg opened this issue 1 year ago • 1 comments

I added a configuration option for OpenELM 1b Instruct 8-bit like so:

  public static let openelm1bInstruct8bit = ModelConfiguration(
    id: "mlx-community/OpenELM-1_1B-Instruct-8bit"
  ) { prompt in
    "\(prompt)"
  }

But when I run the model, I get the following error: Failed: unhandledKeys(base: "Embedding", keys: ["biases", "scales"])

DePasqualeOrg avatar May 07 '24 20:05 DePasqualeOrg

I think this is the same as #53 -- I am still preparing the code to load these new quantized values.

davidkoski avatar May 08 '24 15:05 davidkoski

#76 should fix this

davidkoski avatar May 28 '24 23:05 davidkoski