Trying to load the gpt4chan model

[ksylvan]

Describe the bug

Followed the instructions in the README.md to set up the gpt4chan files:

$ ls -l models/gpt4chan_model_float16/
total 11822356
-rwxrwxrwx 1 kayvan kayvan         930 Apr 27 22:45 config.json
-rwxrwxrwx 1 kayvan kayvan        1482 Jun 12  2022 gpt4chan_model_float16_files.xml
-rwxrwxrwx 1 kayvan kayvan       20480 Jun 12  2022 gpt4chan_model_float16_meta.sqlite
-rwxrwxrwx 1 kayvan kayvan        1231 Jun 12  2022 gpt4chan_model_float16_meta.xml
-rwxrwxrwx 1 kayvan kayvan         741 Apr 27 22:18 index.html
-rwxrwxrwx 1 kayvan kayvan 12106053103 Jun 12  2022 pytorch_model.bin

When I load the model, here's the error message I see:

Traceback (most recent call last):
File “/mnt/c/Users/kayva/src/text-generation-webui/server.py”, line 102, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name)
File “/mnt/c/Users/kayva/src/text-generation-webui/modules/models.py”, line 248, in load_model
tokenizer = AutoTokenizer.from_pretrained(Path(f"{shared.args.model_dir}/{model_name}/"), trust_remote_code=trust_remote_code)
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py”, line 702, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/tokenization_utils_base.py”, line 1795, in from_pretrained
raise EnvironmentError(
OSError: Can’t load tokenizer for ‘models/gpt4chan_model_float16’. If you were trying to load it from ‘https://huggingface.co/models’, make sure you don’t have a local directory with the same name. Otherwise, make sure ‘models/gpt4chan_model_float16’ is the correct path to a directory containing all relevant files for a GPT2TokenizerFast tokenizer.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Followed the GPT-4Chan related instructions in the README.

Direct downloaded the 16-bit files.

After downloading the model, follow these steps:

Place the files under models/gpt4chan_model_float16 or models/gpt4chan_model.

Place GPT-J 6B's config.json file in that same folder: config.json.

Download GPT-J 6B's tokenizer files (they will be automatically detected when you attempt to load GPT-4chan):

python download-model.py EleutherAI/gpt-j-6B --text-only

Screenshot

No response

Logs

Traceback (most recent call last):
File “/mnt/c/Users/kayva/src/text-generation-webui/server.py”, line 102, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name)
File “/mnt/c/Users/kayva/src/text-generation-webui/modules/models.py”, line 248, in load_model
tokenizer = AutoTokenizer.from_pretrained(Path(f"{shared.args.model_dir}/{model_name}/"), trust_remote_code=trust_remote_code)
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py”, line 702, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/tokenization_utils_base.py”, line 1795, in from_pretrained
raise EnvironmentError(
OSError: Can’t load tokenizer for ‘models/gpt4chan_model_float16’. If you were trying to load it from ‘https://huggingface.co/models’, make sure you don’t have a local directory with the same name. Otherwise, make sure ‘models/gpt4chan_model_float16’ is the correct path to a directory containing all relevant files for a GPT2TokenizerFast tokenizer.

System Info

SYSTEM INFO

Running Windows 10 on the hardware below with NVIDIA Geo Force RTX 4090.

(About PC)

Processor       13th Gen Intel(R) Core(TM) i9-13900K   3.00 GHz
Installed RAM   64.0 GB (63.8 GB usable)
Device ID       5758617A-1944-4B90-8C83-EB48B4DD6D85
Product ID      00325-96486-21959-AAOEM
System type     64-bit operating system, x64-based processor
Pen and touch   No pen or touch input is available for this display

GPU INFO:

NVIDIA System Information report created on: 04/22/2023 10:18:05
System name: DESKTOP-8HJQF90

[Display]
Operating System:       Windows 10 Home, 64-bit
DirectX version:        12.0
GPU processor:          NVIDIA GeForce RTX 4090
Driver version:         531.29
Driver Type:            DCH
Direct3D feature level: 12_1
CUDA Cores:             16384
Resizable BAR           Yes
Core clock:             2535 MHz
Memory data rate:       21.00 Gbps
Memory interface:       384-bit
Memory bandwidth:       1.01 TB/s
Total available graphics memory:        57212 MB
Dedicated video memory: 24564 MB GDDR6X
System video memory:    0 MB
Shared system memory:   32648 MB
Video BIOS version:     95.02.18.80.5F
IRQ:                    Not used
Bus:                    PCI Express x16 Gen4
Error Correction Code (ECC):    Off
Device Id:              10DE 2684 40BF1458
Part Number:            G139 0330

[Components]

nvui.dll                8.17.15.3129            NVIDIA User Experience Driver Component
nvxdplcy.dll            8.17.15.3129            NVIDIA User Experience Driver Component
nvxdbat.dll             8.17.15.3129            NVIDIA User Experience Driver Component
nvxdapix.dll            8.17.15.3129            NVIDIA User Experience Driver Component
NVCPL.DLL               8.17.15.3129            NVIDIA User Experience Driver Component
nvCplUIR.dll            8.1.940.0               NVIDIA Control Panel
nvCplUI.exe             8.1.940.0               NVIDIA Control Panel
nvWSSR.dll              31.0.15.3129            NVIDIA Workstation Server
nvWSS.dll               31.0.15.3129            NVIDIA Workstation Server
nvViTvSR.dll            31.0.15.3129            NVIDIA Video Server
nvViTvS.dll             31.0.15.3129            NVIDIA Video Server
nvLicensingS.dll                6.14.15.3129            NVIDIA Licensing Server
nvDevToolSR.dll         31.0.15.3129            NVIDIA Licensing Server
nvDevToolS.dll          31.0.15.3129            NVIDIA 3D Settings Server
nvDispSR.dll            31.0.15.3129            NVIDIA Display Server
nvDispS.dll             31.0.15.3129            NVIDIA Display Server
PhysX           09.21.0713              NVIDIA PhysX
NVCUDA64.DLL            31.0.15.3129            NVIDIA CUDA 12.1.68 driver
nvGameSR.dll            31.0.15.3129            NVIDIA 3D Settings Server
nvGameS.dll             31.0.15.3129            NVIDIA 3D Settings Server

[sourabhv]

Same error on MacOS M1 Pro

Loading gpt4chan_model_float16...
Traceback (most recent call last):
  File "/Users/foo/Downloads/text-generation-webui/text-generation-webui/server.py", line 914, in <module>
    shared.model, shared.tokenizer = load_model(shared.model_name)
  File "/Users/foo/Downloads/text-generation-webui/text-generation-webui/modules/models.py", line 248, in load_model
    tokenizer = AutoTokenizer.from_pretrained(Path(f"{shared.args.model_dir}/{model_name}/"), trust_remote_code=trust_remote_code)
  File "/Users/foo/.pyenv/versions/3.10.5/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 702, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/Users/foo/.pyenv/versions/3.10.5/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1795, in from_pretrained
    raise EnvironmentError(
OSError: Can't load tokenizer for 'models/gpt4chan_model_float16'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'models/gpt4chan_model_float16' is the correct path to a directory containing all relevant files for a GPT2TokenizerFast tokenizer.

[gitmylo]

Solution:

download GPT-2's tokenizer.json and place it in your GPT4Chan directory, gpt4chan_model_float16 in my case.

or simply open a cmd in your webui directory and run:

curl -o models\gpt4chan_model_float16\tokenizer.json https://huggingface.co/gpt2/raw/main/tokenizer.json

(make sure you change gpt4chan_model_float16 to the name it has in your models folder)

ps: if you encounter issues with missing tokens or something, maybe this one will work better, as that's the tokenizer for GPT-J-6B

[ksylvan]

Thanks for the reply, but still having issues (slightly different now).

I had put the tokenizer.json in the directory as mentioned in the README, but I've now replaced it with the one from huggingface, and here's the new error I'm seeing:

Traceback (most recent call last):
File “/mnt/c/Users/kayvan/src/text-generation-webui/server.py”, line 60, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name)
File “/mnt/c/Users/kayvan/src/text-generation-webui/modules/models.py”, line 250, in load_model
tokenizer = AutoTokenizer.from_pretrained(Path(f"{shared.args.model_dir}/{model_name}/"), trust_remote_code=trust_remote_code)
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py”, line 692, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/tokenization_utils_base.py”, line 1812, in from_pretrained
return cls._from_pretrained(
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/tokenization_utils_base.py”, line 1975, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/gpt2/tokenization_gpt2_fast.py”, line 140, in init
super().init(
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py”, line 117, in init
slow_tokenizer = self.slow_tokenizer_class(*args, **kwargs)
File “/home/kayvan/.miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/gpt2/tokenization_gpt2.py”, line 194, in init
with open(merges_file, encoding=“utf-8”) as merges_handle:
TypeError: expected str, bytes or os.PathLike object, not NoneType

[ksylvan]

I got this to work by doing this:

1. Set up the gpt4chan_model_float16 folder using the downloaded files, as described in the README.
2. python download-model.py EleutherAI/gpt-j-6B --text-only
3. cd models/ gpt4chan_model_float16/; ln -sf ../EleutherAI_gpt-j-6B/* .

We can close this.