Syntax/Logic error? pad_token is used before it is defined.

[phalexo]

DEFAULT_PAD_TOKEN is set to a string "[PAD]"

Under Python 3.10 this code dict(pad_token=DEFAULT_PAD_TOKEN), causes an exception. I tried to modify it to dict({"unk_token":pad_token}) setting pad_token earlier to DEFAULT_PAD_TOKEN, but it causes various other issues. "[PAD]" is not defined as a special token, etc...

if tokenizer.pad_token is None:
    smart_tokenizer_and_embedding_resize(
        special_tokens_dict=dict(pad_token=DEFAULT_PAD_TOKEN),
        tokenizer=tokenizer,
        model=model,
    )

[phalexo]

trainable params: 79953920.0 || all params: 3660320768 || trainable: 2.184341894267557 loaded model Using pad_token, but it is not set yet. pad_token_id = -1

Traceback (most recent call last): File "/home/developer/qlora/qlora.py", line 773, in train() File "/home/developer/qlora/qlora.py", line 635, in train "unk_token": tokenizer.convert_ids_to_tokens(model.config.pad_token_id), File "/home/developer/mambaforge/envs/Guanaco/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 307, in convert_ids_to_tokens return self._tokenizer.id_to_token(ids) OverflowError: out of range integral type conversion attempted