Exception handling method when there is no pad token in tokenizer

[idisuu]

https://github.com/LAION-AI/Open-Assistant/blob/0fcf3e08fe62295d4696e590005b0f33383342ea/model/model_training/trainer_rl.py#L77-L83

In reinforcement learning, there is a part that converts the pad token of the sft tokenizer to the pad token of the rank tokenizer.

However, pad tokens do not exist in all tokenizers.

For example, the tokenizer of HuggingFace's "EleutherAI/pythia-70m-deduped" used for debugging does not have a pad token, so the following error occurs.

This is because the pad token does not exist in the tokenizer as shown below.

Therefore, exception handling is required for the tokenizer where the pad token does not exist for the code.

[BrianPulfer]

I'm interested in helping solving this issue. How should the exception be handled? Using other tokens instead (like unk)? Or just providing an error message instead?

[idisuu]

I'm interested in helping solving this issue. How should the exception be handled? Using other tokens instead (like unk)? Or just providing an error message instead?

In my personal opinion, it seems that the part where the pad_token of sft_tokenizer is converted to the pad_token of rank_tokenizer can be handled with a conditional statement.

For example

if sft_tokenizer.pad_token:
    samples = [x.replace(sft_tokenizer.pad_token, rank_tokenizer.pad_token) for x in samples] 

In my experience, when there is no pad_token, the eos_token typically serves as the pad_token.

Therefore, if there is no pad_token, I believe that replacing the eos_token of sft_tokenizer with the eos_token of rank_tokenizer would have the same logical effect as changing the pad_token.

Using a conditional statement would indeed solve the immediate problem, I believe. However, when I ran the training, the mentioned code segment was executed during rollout method in the trlx library after loading both the model and the data onto the GPU. This means that I had to wait for about 30 minutes to see the error in that part.

Therefore, in my opinion, it would be more user-friendly to handle the error proactively before loading the model and data onto the GPU.

However, I'm not exactly sure about the philosophy that Open-Assistant specifically pursues.