[Feature] GPU support for GPT-J

[3Simplex]

Bug Report

When using Mac set to use metal, gpt-j model fails to fallback to CPU.

Steps to Reproduce

1. With a Mac set application device to use metal.
2. Try to load a gpt-j architecture model.
3. With a Mac set application device to use CPU.
4. Load the same gpt-j architecture model.

Expected Behavior

The gpt-j model has no GPU support should fallback to CPU.

Your Environment

• GPT4All version: 2.8.0
• Operating System: Mac
• Chat model used (if applicable): https://huggingface.co/EleutherAI/gpt-j-6b converted to gguf.

Works in windows as expected with Vulkan.

[3Simplex]

From Discord: qui — Today at 7:13 PM Yes. Happened when set to auto as well - in MAC Also failed when set to CUDA on Windows.

[cebtenzzre]

GPU support for GPT-J would basically require upstreaming it into llama.cpp. It's the one architecture that we still support using an old implementation from the original ggml repo.

[cebtenzzre]

Support for GPT-J models was removed in #2676. If someone makes a proper llama.cpp implementation of GPT-J, it should get GPU support automatically.