gpt4all icon indicating copy to clipboard operation
gpt4all copied to clipboard

Can't change GPU Layers setting for any model with a number larger than default

Open Flu opened this issue 1 year ago • 1 comments

Bug Report

When going into Settings -> Model (any model will do), changing the "GPU Layers" setting is impossible for numbers larger than the default. It always reverts back to 32 for Llama 3.1 8B Instruct 128k, for example. Morever, if you click on "Restore defaults", it writes 100 to the textbox, no matter the model or the default value, as far as I can tell.

Steps to Reproduce

  1. Go to Settings -> Models
  2. Select any model and scroll down
  3. Click inside the text box for the setting "GPU Layers" and input any number larger than the default
  4. Click on any other textbox inside the menu or exit the application entirely
  5. The value reverts back to the default
  6. Click on "Revert Defaults" and the value is now 100.
  7. If the application is restarted, the value is again 32, which is indeed the default.

Expected Behavior

The application should let me change the number to something larger than the default. If the behaviour is meant to signify that 32 is the maximum number of GPU layers possible for this model, this should be clearly stated there as being the max. But even so, clicking the "Restore defaults" should revert the value back to 32, not 100.

Your Environment

  • GPT4All version: 3.1.1
  • Operating System: Windows 11 Pro 23H2
  • Chat model used (if applicable): Any

Flu avatar Aug 07 '24 14:08 Flu