gpt4all icon indicating copy to clipboard operation
gpt4all copied to clipboard

[Feature] GPU support for GPT-J

Open 3Simplex opened this issue 8 months ago • 2 comments

Bug Report

When using Mac set to use metal, gpt-j model fails to fallback to CPU.

Steps to Reproduce

  1. With a Mac set application device to use metal.
  2. Try to load a gpt-j architecture model.
  3. With a Mac set application device to use CPU.
  4. Load the same gpt-j architecture model.

Expected Behavior

The gpt-j model has no GPU support should fallback to CPU.

Your Environment

  • GPT4All version: 2.8.0
  • Operating System: Mac
  • Chat model used (if applicable): https://huggingface.co/EleutherAI/gpt-j-6b converted to gguf.

Works in windows as expected with Vulkan.

3Simplex avatar Jun 12 '24 23:06 3Simplex