gpt4all icon indicating copy to clipboard operation
gpt4all copied to clipboard

[Feature] Crash: Support old MPT GGUF conversions with duplicated output tensor

Open dlippold opened this issue 1 year ago • 2 comments

Bug Report

The fine-tuned MPT model from https://huggingface.co/maddes8cht/mosaicml-mpt-7b-instruct-gguf/ in quantization Q4_1 was usabel in release 2.7.2 but not longer in 2.7.3 and later. In particular it is currently not usable.

When I try to load the model file I get the following error message:

Could not load model due to invalid model file for mosaicml-mpt-7b-instruct-Q4_1.gguf

The reason of the problem may have to do with #2006

Steps to Reproduce

  1. Download the model file from the specified URL
  2. Start GPT4all
  3. Choose the downloaded model file

Expected Behavior

The model file should be loaded.

Your Environment

  • GPT4All version: 2.7.2, 2.7.3, 2.7.5
  • Operating System: Ubuntu Linux 22.04.
  • Chat model used (if applicable): see above

dlippold avatar May 09 '24 17:05 dlippold

I fixed this upstream in https://github.com/ggerganov/llama.cpp/pull/6139 which should make it into the next release of GPT4All (already included in #2310).

cebtenzzre avatar May 09 '24 22:05 cebtenzzre

Version 2.8.0 crashes when loading the model named above.

dlippold avatar Jun 29 '24 10:06 dlippold