[Feature] Ability to check for updates to discovered models

[cebtenzzre]

Discussed in https://github.com/nomic-ai/gpt4all/discussions/2064

^{Originally posted by SINAPSA-IC March 2, 2024} Hello.

Assuming that there's an update for an LLM that we use in GPT4All, how can we know this? Will such an update be listed under "Updates -> components", or under "Downloads"? Will the text (and functionality ofc) of the "Download" button change to "Update" if a check is being performed - preferably, at the user's request? Although I guess the only feasible option is to download a certain quantization, adapted for use within the program...

Thank you.

[manyoso]

There exists no way to know this currently. OTOH, the way 'updates' usually happen for a model is for a new model to be published under a different name. I can't think of a single instance where this hasn't happened?

[cebtenzzre]

I can't think of a single instance where this hasn't happened?

Usually the weights are not updated. But sometimes the upstream model architecture changes, e.g.:

• https://huggingface.co/tiiuae/falcon-40b/commit/4a70170c215b36a3cce4b4253f6d0612bb7d4146
• https://huggingface.co/replit/replit-code-v1-3b/commit/e023a8461c7a2e2f2c85e52dbdd8d68b415f95eb

And other times, the GGUF conversion needs to be updated for a number of reasons:

• llama.cpp updated the quantization for decreased perplexity:
  • sometimes backwards-compatible: https://www.github.com/ggerganov/llama.cpp/pull/2707
  • sometimes backwards-incompatible (allegedly): https://www.github.com/ggerganov/llama.cpp/pull/5829
• llama.cpp changed the GGUF metadata in some way:
  • changes to tokenization, e.g. this breaking change https://www.github.com/ggerganov/llama.cpp/pull/3252
  • changes for special tokens, e.g. the reason that we had to re-convert Mistral OpenOrca
  • new keys, e.g. tokenizer.chat_template: https://www.github.com/ggerganov/llama.cpp/pull/4125

If the conversion is updated, a new commit is made to the repo - they are not immutable. Ideally there would be a way to check for this in the UI, by comparing the downloaded model's hash against the latest etag on the remote. Maybe there could be a "check for updates" button, or maybe it would be done automatically on startup. Then users could click "Update" for any models that have changed.